- [newgrounds] add
search
extractor (#2161)
- restore
-d/--dest
functionality from before 1.20.0 (#2148) - change short option for
--directory
to-D
- [gelbooru] handle changed API response format (#2157)
- [hitomi] fix image URLs (#2153)
- [mangadex] fix extraction (#2177)
- [rule34] use
https://api.rule34.xxx
for API requests - fix cookie checks for patreon, fanbox, fantia
- improve UNC path handling (#2126)
- [500px] add
favorite
extractor (#1927) - [exhentai] add
source
option - [fanbox] support pixiv redirects (#2122)
- [inkbunny] add
search
extractor (#2094) - [kemonoparty] support coomer.party (#2100)
- [lolisafe] add generic album extractor for lolisafe/chibisafe instances (#2038, #2105)
- [rule34us] add
tag
andpost
extractors (#1527) - add a generic extractor (#735, #683)
- add
-d/--directory
and-f/--filename
command-line options - add
--sleep-request
and--sleep-extractor
command-line options - allow specifying
sleep-*
options as string
- [cyberdrop] include file ID in default filenames
- [hitomi] disable
metadata
by default - [kemonoparty] use
service
as subcategory (#2147) - [kemonoparty] change default
files
order toattachments,file,inline
(#1991) - [output] write download progress indicator to stderr
- [ytdl] prefer yt-dlp over youtube-dl (#1850, #2028)
- rename
--write-infojson
to--write-info-json
- [500px] create directories per photo
- [artstation] create directories per asset (#2136)
- [deviantart] use
/browse/newest
for most-recent searches (#2096) - [hitomi] fix image URLs
- [instagram] fix error when PostPage data is not in GraphQL format (#2037)
- [instagran] match post URLs with usernames (#2085)
- [instagram] allow downloading specific stories (#2088)
- [furaffinity] warn when no session cookies were found
- [pixiv] respect date ranges in search URLs (#2133)
- [sexcom] fix and improve embed extraction (#2145)
- [tumblrgallery] fix extraction (#2112)
- [tumblrgallery] improve
id
extraction (#2115) - [tumblrgallery] improve search pagination (#2132)
- [twitter] include
4096x4096
as a default image fallback (#1881, #2107) - [ytdl] update argument parsing to latest yt-dlp changes (#2124)
- handle UNC paths (#2113)
- [dynastyscans] add
manga
extractor (#2035) - [instagram] include user metadata for
tagged
downloads (#2024) - [kemonoparty] implement
files
option (#1991) - [kemonoparty] add
dms
option (#2008) - [mangadex] always provide
artist
,author
, andgroup
metadata fields (#2049) - [philomena] support furbooru.org (#1995)
- [reactor] support thatpervert.com (#2029)
- [shopify] support loungeunderwear.com (#2053)
- [skeb] add
thumbnails
option (#2047, #2051) - [subscribestar] add
num
enumeration index (#2040) - [subscribestar] emit metadata for posts without media (#1569)
- [ytdl] implement
cmdline-args
andconfig-file
options to allow parsing ytdl command-line options (#1680) - [formatter] implement
D
format specifier - extend
blacklist
/whitelist
syntax (#2025)
- [dynastyscans] provide
date
as datetime object (#2050) - [exhentai] fix extraction for disowned galleries (#2055)
- [gelbooru] apply workaround for pagination limits
- [kemonoparty] skip duplicate files (#2032, #1991, #1899)
- [kemonoparty] provide
date
metadata for gumroad (#2007) - [mangoxo] fix metadata extraction
- [twitter] distinguish between fatal & nonfatal errors (#2020)
- [twitter] fix extractor for direct image links (#2030)
- [webtoons] use download URLs that do not require a
Referer
header (#2005) - [ytdl] improve error handling (#1680)
- [downloader:ytdl] prevent crash in
_progress_hook()
(#1680)
- [seisoparty] remove module
- [kemonoparty] add
comments
option (#1980) - [skeb] add
user
andpost
extractors (#1031, #1971) - [twitter] add
pinned
option - support accessing environment variables and the current local datetime in format strings (#1968)
- add special type format strings to docs (#1987)
- [cyberdrop] fix video extraction (#1993)
- [deviantart] fix
index
values for stashed deviations - [gfycat] provide consistent
userName
values foruser
downloads (#1962) - [gfycat] show warning when there are no available formats
- [hitomi] fix image URLs (#1975, #1982, #1988)
- [instagram] update query hashes
- [mangakakalot] update domain and fix extraction
- [mangoxo] fix login and extraction
- [reddit] prevent crash for galleries with no
media_metadata
(#2001) - [redgifs] update to API v2 (#1984)
- fix calculating retry sleep times (#1990)
- [inkbunny] add
following
extractor (#515) - [inkbunny] add
pool
extractor (#1937) - [kemonoparty] add
discord
extractor (#1827, #1940) - [nhentai] add
tag
extractor (#1950, #1955) - [patreon] add
files
option (#1935) - [picarto] add
gallery
extractor (#1931) - [pixiv] add
sketch
extractor (#1497) - [seisoparty] add
favorite
extractor (#1906) - [twitter] add
size
option (#1881) - [vk] add
album
extractor (#474, #1952) - [postprocessor:compare] add
equal
option (#1592)
- [cyberdrop] extract direct download URLs (#1943)
- [deviantart] update
search
argument handling (#1911) - [deviantart] full resolution for non-downloadable images (#293)
- [furaffinity] unquote search queries (#1958)
- [inkbunny] match "long" URLs for pools and favorites (#1937)
- [kemonoparty] improve inline extraction (#1899)
- [mangadex] update parameter handling for API requests (#1908)
- [patreon] better filenames for
content
images (#1954) - [redgifs][gfycat] provide fallback URLs (#1962)
- [downloader:ytdl] prevent crash in
_progress_hook()
- restore SOCKS support for Windows executables
- [aryion] add
tag
extractor (#1849) - [desktopography] implement desktopography extractors (#1740)
- [deviantart] implement
auto-unwatch
option (#1466, #1757) - [fantia] add
date
metadata field (#1853) - [fappic] add
image
extractor (#1898) - [gelbooru_v02] add
favorite
extractor (#1834) - [kemonoparty] add
favorite
extractor (#1824) - [kemonoparty] implement login with username & password (#1824)
- [mastodon] add
following
extractor (#1891) - [mastodon] support specifying accounts by ID
- [twitter] support
/with_replies
URLs (#1833) - [twitter] add
quote_by
metadata field (#1481) - [postprocessor:compare] extend
action
option (#1592) - implement a download progress indicator (#1519)
- implement a
page-reverse
option (#1854) - implement a way to specify extended format strings
- allow specifying a minimum/maximum for
sleep-*
options (#1835) - add a
--write-infojson
command-line option
- [cyberdrop] change directory name format (#1871)
- [instagram] update default delay to 6-12 seconds (#1835)
- [reddit] extend subcategory depending on input URL (#1836)
- move util.Formatter and util.PathFormat into their own modules
- [artstation] use
/album/all
view for user portfolios (#1826) - [aryion] update/improve pagination (#1849)
- [deviantart] fix bug with fetching premium content (#1879)
- [deviantart] update default archive_fmt for single deviations (#1874)
- [erome] send Referer header for file downloads (#1829)
- [hiperdex] fix extraction
- [kemonoparty] update file download URLs (#1902, #1903)
- [mangadex] fix extraction (#1852)
- [mangadex] fix retrieving chapters from "pornographic" titles (#1908)
- [nozomi] preserve case of search tags (#1860)
- [redgifs][gfycat] remove webtoken code (#1907)
- [twitter] ensure card entries have a
url
(#1868) - implement a way to correctly shorten displayed filenames containing east-asian characters (#1377)
- [420chan] add
thread
andboard
extractors (#1773) - [deviantart] add
tag
extractor (#1803) - [deviantart] add
comments
option (#1800) - [deviantart] implement a
auto-watch
option (#1466, #1757) - [foolfuuka] add
gallery
extractor (#1785) - [furaffinity] expand URL pattern for searches (#1780)
- [kemonoparty] automatically generate required DDoS-GUARD cookies (#1779)
- [nhentai] add
favorite
extractor (#1814) - [shopify] support windsorstore.com (#1793)
- [twitter] add
url
to user objects (#1787, #1532) - [twitter] expand t.co links in user descriptions (#1787, #1532)
- show a warning if an extractor doesn`t yield any results (#1428, #1759)
- add a
j
format string conversion - implement a
fallback
option (#1770) - implement a
path-strip
option
- [shopify] use API for product listings (#1793)
- update default User-Agent headers
- [deviantart] prevent exceptions for "empty" videos (#1796)
- [exhentai] improve image limits check (#1808)
- [inkbunny] fix extraction (#1816)
- [mangadex] prevent exceptions for manga without English title (#1815)
- [oauth] use defaults when config values are set to
null
(#1778) - [pixiv] fix pixivision title extraction
- [reddit] delay RedditAPI initialization (#1813)
- [twitter] improve error reporting (#1759)
- [twitter] fix issue when filtering quote tweets (#1792)
- [twitter] fix
logout
option (#1719)
- [deviantart] remove the "you need session cookies to download mature scraps" warning (#1777, #1776)
- [foolslide] remove entry for kobato.hologfx.com
- [bbc] add
width
option (#1706) - [danbooru] add
external
option (#1747) - [furaffinity] add
external
option (#1492) - [luscious] add
gif
option (#1701) - [newgrounds] add
format
option (#1729) - [reactor] add
gif
option (#1701) - [twitter] warn about suspended accounts (#1759)
- [twitter] extend
replies
option (#1254) - [twitter] add option to log out and retry when blocked (#1719)
- [wikieat] add
thread
andboard
extractors (#1699, #1607)
- [instagram] increase default delay between HTTP requests from 5s to 8s (#1732)
- [bbc] improve image dimensions (#1706)
- [bbc] support multi-page gallery listings (#1730)
- [behance] fix
collection
extraction - [deviantart] get original files for GIF previews (#1731)
- [furaffinity] fix errors when using
category-transfer
(#1274) - [hitomi] fix image URLs (#1765)
- [instagram] use custom User-Agent header for video downloads (#1682, #1623, #1580)
- [kemonoparty] fix username extraction (#1750)
- [kemonoparty] update file server domain (#1764)
- [newgrounds] fix errors when using
category-transfer
(#1274) - [nsfwalbum] retry backend requests when extracting image URLs (#1733, #1271)
- [vk] prevent exception for empty/private profiles (#1742)
- [bbc] add
gallery
andprogramme
extractors (#1706) - [comicvine] add extractor (#1712)
- [kemonoparty] add
max-posts
option (#1674) - [kemonoparty] parse
o
query parameters (#1674) - [mastodon] add
reblogs
andreplies
options (#1669) - [pixiv] add extractor for
pixivision
articles (#1672) - [ytdl] add experimental extractor for sites supported by youtube-dl (#1680, #878)
- extend
parent-metadata
functionality (#1687, #1651, #1364) - add
archive-prefix
option (#1711) - add
url-metadata
option (#1659, #1073)
- [kemonoparty] skip duplicated patreon files (#1689, #1667)
- [mangadex] use custom User-Agent header (#1535)
- [hitomi] fix image URLs (#1679)
- [imagevenue] fix extraction (#1677)
- [instagram] fix extraction of
/explore/tags/
posts (#1666) - [moebooru] fix
tags
ending with a+
when logged in (#1702) - [naverwebtoon] fix comic extraction
- [pururin] update domain and fix extraction
- [vk] improve metadata extraction and URL pattern (#1691)
- [downloader:ytdl] fix
outtmpl
setting for yt-dlp (#1680)
- [mangafox] add
manga
extractor (#1633) - [mangasee] add
chapter
andmanga
extractors - [mastodon] implement
text-posts
option (#1569, #1669) - [seisoparty] add
user
andpost
extractors (#1635) - implement conditional directories (#1394)
- add
T
format string conversion (#1646) - document format string syntax
- [twitter] set
retweet_id
for original retweets (#1481)
- [directlink] manually encode Referer URLs (#1647)
- [hiperdex] use domain from input URL
- [kemonoparty] fix
username
extraction (#1652) - [kemonoparty] warn about missing DDoS-GUARD cookies
- [twitter] ensure guest tokens are returned as string (#1665)
- [webtoons] match arbitrary language codes (#1643)
- fix depth counter in UrlJob when specifying
-g
multiple times
- [foolfuuka] support
archive.wakarimasen.moe
(#1595) - [mangadex] implement login with username & password (#1535)
- [mangadex] add extractor for a user's followed feed (#1535)
- [pixiv] support fetching privately followed users (#1628)
- implement conditional filenames (#1394)
- implement
filter
option for post processors (#1460) - add
-T/--terminate
command-line option (#1399) - add
-P/--postprocessor
command-line option (#1583)
- [kemonoparty] update default filenames and archive IDs (#1514)
- [twitter] update default settings
- change
retweets
andquoted
options fromtrue
tofalse
- change directory format for search results to the same as other extractors
- change
- require an argument for
--clear-cache
- [500px] update GraphQL queries
- [furaffinity] improve metadata extraction (#1630)
- [hitomi] update image URL generation (#1637)
- [idolcomplex] improve and fix pagination (#1594, #1601)
- [instagram] fix login (#1631)
- [instagram] update query hashes
- [mangadex] update to API v5 (#1535)
- [mangafox] improve URL pattern (#1608)
- [oauth] prevent exceptions when reporting errors (#1603)
- [philomena] fix tag escapes handling (#1629)
- [redgifs] update API server address (#1632)
- [sankaku] handle empty tags (#1617)
- [subscribestar] improve attachment filenames (#1609)
- [unsplash] update collections URL pattern (#1627)
- [postprocessor:metadata] handle dicts in
mode:tags
(#1598)
- [kemonoparty] add
metadata
option (#1548) - [kemonoparty] add
type
metadata field (#1556) - [mangapark] recognize v2.mangapark URLs (#1578)
- [patreon] extract user-defined
tags
(#1539, #1540) - [pillowfort] implement login with username & password (#846)
- [pillowfort] add
inline
andexternal
options (#846) - [pixiv] implement
max-posts
option (#1558) - [pixiv] add
metadata
option (#1551) - [twitter] add
text-tweets
option (#570) - [weibo] extend
retweets
option (#1542) - [postprocessor:ugoira] support using the
image2
demuxer (#1550) - [postprocessor:ugoira] add
repeat-last-frame
option (#1550) - support
XDG_CONFIG_HOME
(#1545) - implement
parent-skip
and"skip": "terminate"
options (#1399)
- [twitter] resolve
t.co
URLs incontent
(#1532)
- [500px] update query hashes (#1573)
- [aryion] find text posts in
recursive=false
mode (#1568) - [imagebam] fix extraction of NSFW images (#1534)
- [imgur] update URL patterns (#1561)
- [manganelo] update domain to
manganato.com
- [reactor] skip deleted/empty posts
- [twitter] add missing retweet media entities (#1555)
- fix ISO 639-1 code for Japanese (
jp
->ja
)
- [gelbooru] add extractor for
/redirect.php
URLs (#1530) - [inkbunny] add
favorite
extractor (#1521) - add
output.skip
option - add an optional argument to
--clear-cache
to select which cache entries to remove (#1230)
- [pixiv] update
translated-tags
option (#1507)- rename to
tags
- accept
"japanese"
,"translated"
, and"original"
as values
- rename to
- [500px] update query hashes
- [kemonoparty] fix download URLs (#1514)
- [imagebam] fix extraction
- [instagram] update query hashes
- [nozomi] update default archive-fmt for
tag
andsearch
extractors (#1529) - [pixiv] remove duplicate translated tags (#1507)
- [readcomiconline] change domain to
readcomiconline.li
(#1517) - [sankaku] update invalid-token detection (#1515)
- fix crash when using
--no-download
with--ugoira-conv
(#1507)
- [danbooru] add option for extended metadata extraction (#1458)
- [fanbox] add extractors (#1459)
- [fantia] add extractors (#1459)
- [gelbooru] add an option to extract notes (#1457)
- [hentaicosplays] add extractor (#907, #1473, #1483)
- [instagram] add extractor for
tagged
posts (#1439) - [naverwebtoon] ignore non-comic images
- [pixiv] also save untranslated tags when
translated-tags
is enabled (#1501) - [shopify] support omgmiamiswimwear.com (#1280)
- implement
output.fallback
option - add archive format to InfoJob output (#875)
- build executables with SOCKS proxy support (#1424)
- [500px] update query hashes
- [8muses] fix JSON deobfuscation
- [artstation] download
/4k/
images (#1422) - [deviantart] fix pagination for Eclipse results (#1444)
- [deviantart] improve folder name matching (#1451)
- [erome] skip deleted albums (#1447)
- [exhentai] fix image limit detection (#1437)
- [exhentai] restore
limits
option (#1487) - [gelbooru] fix tag category extraction (#1455)
- [instagram] update query hashes
- [komikcast] fix extraction
- [simplyhentai] fix extraction
- [slideshare] fix extraction
- [webtoons] update agegate/GDPR cookies (#1431)
- fix
category-transfer
option
- [yuki] remove module for yuki.la
- [deviantart] add support for posts from watched users (#794)
- [manganelo] add
chapter
andmanga
extractors (#1415) - [pinterest] add
search
extractor (#1411) - [sankaku] add
tag_string
metadata field (#1388) - [sankaku] add enumeration index for books (#1388)
- [tapas] add
series
andepisode
extractors (#692) - [tapas] implement login with username & password (#692)
- [twitter] allow specifying a custom format for user results (#1337)
- [twitter] add extractor for direct image links (#1417)
- [vk] add support for albums (#474)
- [aryion] unescape paths (#1414)
- [bcy] improve pagination
- [deviantart] update
watch
URL pattern (#794) - [deviantart] fix arguments for search/popular results (#1408)
- [deviantart] use fallback for
/intermediary/
URLs - [exhentai] improve and simplify image limit checks
- [komikcast] fix extraction
- [pixiv] fix
favorite
URL pattern (#1405) - [sankaku] simplify
pool
tags (#1388) - [twitter] improve error message when trying to log in with 2FA (#1409)
- [twitter] don't use youtube-dl for cards when videos are disabled (#1416)
- [architizer] add
project
andfirm
extractors (#1369) - [deviantart] add
watch
extractor (#794) - [exhentai] support
/tag/
URLs (#1363) - [gelbooru_v01] support
drawfriends.booru.org
,vidyart.booru.org
, andtlb.booru.org
by default - [nozomi] support
/index-N.html
URLs (#1365) - [philomena] add generalized extractors for philomena sites (#1379)
- [philomena] support post URLs without
/images/
- [twitter] implement
users
option (#1337) - implement
parent-metadata
option (#1364)
- [exhentai] improve favorites count extraction (#1360)
- [gelbooru] update domain for video downloads (#1368)
- [hentaifox] improve image and metadata extraction (#1366, #1378)
- [imgur] fix and improve rate limit handling (#1386)
- [weasyl] improve favorites URL pattern (#1374)
- use type check before applying
browser
option (#1358) - ensure
-s/--simulate
always prints filenames (#1360)
- [hentaicafe] remove module
- [hentainexus] remove module
- [mangareader] remove module
- [mangastream] remove module
- [cyberdrop] add support for
https://cyberdrop.me/
(#1328) - [exhentai] add
metadata
option; extract more metadata from gallery pages (#1325) - [hentaicafe] add
search
andtag
extractors (#1345) - [hentainexus] add
original
option (#1322) - [instagram] support
/user/reels/
URLs (#1329) - [naverwebtoon] add support for
https://comic.naver.com/
(#1331) - [pixiv] add
translated-tags
option (#1354) - [tbib] add support for
https://tbib.org/
(#473, #1082) - [tumblrgallery] add support for
https://tumblrgallery.xyz/
(#1298) - [twitter] add extractor for followed users (#1337)
- [twitter] add option to download all media from conversations (#1319)
- [wallhaven] add
collections
extractor (#1351) - [snap] allow access to user's .netrc for site authentication (#1352)
- add extractors for Gelbooru v0.1 sites (#234, #426, #473, #767, #1238)
- add
-E/--extractor-info
command-line option (#875) - add GitHub Actions workflow for building standalone executables (#1312)
- add
browser
andheaders
options (#1117) - add option to use different youtube-dl forks (#1330)
- support using multiple input files at once (#1353)
- [deviantart] extend
extra
option to also download embedded DeviantArt posts. - [exhentai] rename metadata fields to match API results (#1325)
- [mangadex] use
api.mangadex.org
as default API server - [mastodon] cache OAuth tokens (#616)
- replace
wait-min
andwait-max
withsleep-request
- [500px] skip unavailable photos (#1335)
- [komikcast] fix extraction
- [readcomiconline] download high quality image versions (#1347)
- [twitter] update GraphQL endpoints
- fix crash when
base-directory
is an empty string (#1339)
- remove support for formerly deprecated options
- remove
cloudflare
module
- [behance] support
video
modules (#1282) - [erome] add
album
,user
, andsearch
extractors (#409) - [hentaifox] support searching by group (#1294)
- [imgclick] add
image
extractor (#1307) - [kemonoparty] extract inline images (#1286)
- [kemonoparty] support URLs with non-numeric user and post IDs (#1303)
- [pillowfort] add
user
andpost
extractors (#846)
- [kemonoparty] include
service
in directories and archive keys - [pixiv] require a
refresh-token
to login (#1304) - [snap] use
core18
as base
- [500px] update query hashes
- [deviantart] update parameters for
/browse/popular
(#1267) - [deviantart] provide filename extension for original file downloads (#1272)
- [deviantart] fix
folders
option (#1302) - [inkbunny] add
sid
parameter to private file downloads (#1281) - [kemonoparty] fix absolute file URLs
- [mangadex] revert to
https://mangadex.org/api/
and addapi-server
option (#1310) - [nsfwalbum] use fallback for deleted content (#1259)
- [sankaku] update
invalid token
detection (#1309) - [slideshare] fix extraction
- [postprocessor:metadata] fix crash with
extension-format
(#1285)
- [furaffinity] add
descriptions
option (#1231) - [kemonoparty] add
user
andpost
extractors (#1216) - [nozomi] add
num
enumeration index (#1239) - [photovogue] added portfolio extractor (#1253)
- [twitter] match
/i/user/ID
URLs - [unsplash] add extractors (#1197)
- [vipr] add image extractor (#1258)
- [derpibooru] use "Everything" filter by default (#862)
- [derpibooru] update
date
parsing - [foolfuuka] stop search when results are exhausted (#1174)
- [instagram] fix regex for
/saved
URLs (#1251) - [mangadex] update API URLs
- [mangakakalot] fix extraction
- [newgrounds] fix flash file extraction (#1257)
- [sankaku] simplify login process
- [twitter] fix retries after hitting rate limit
- fix crash when using a
dict
forpath-restrict
- [postprocessor:metadata] sanitize custom filenames
- [derpibooru] add
search
andgallery
extractors (#862) - [foolfuuka] add
board
andsearch
extractors (#1044, #1174) - [gfycat] add
date
metadata field (#1138) - [pinterest] add support for getting all boards of a user (#1205)
- [sankaku] add support for book searches (#1204)
- [twitter] fetch media from pinned tweets (#1203)
- [wikiart] add extractor for single paintings (#1233)
- [downloader:http] add MIME type and signature for
.ico
files (#1211) - add
d
format string conversion for timestamp values - add
"ascii"
as a specialpath-restrict
value
- [hentainexus] fix extraction (#1234)
- [instagram] categorize single highlight URLs as
highlights
(#1222) - [redgifs] fix search results
- [twitter] fix login with username & password
- [twitter] fetch tweets from
homeConversation
entries
- [instagram] add
include
option (#1180) - [pinterest] implement video support (#1189)
- [sankaku] reimplement login support (#1176, #1182)
- [sankaku] add support for sankaku.app URLs (#1193)
- [e621] return pool posts in order (#1195)
- [hentaicafe] prefer title of
/hc.fyi/
pages (#1106) - [hentaicafe] simplify default filenames
- [sankaku] normalize
created_at
metadata (#1190) - [postprocessor:exec] do not add missing
{}
to command (#1185)
- [booru] improve error handling
- [instagram] warn about private profiles (#1187)
- [keenspot] improve redirect handling
- [mangadex] respect
chapter-reverse
settings (#1194) - [pixiv] output debug message on failed login attempts (#1192)
- increase SQLite connection timeouts (#1173)
- [mangapanda] remove module
- [booru] implement generalized extractors for
*booru
andmoebooru
sites - [postprocessor:metadata] add
event
andfilename
options (#315, #866, #984) - [postprocessor:exec] add
event
option (#992)
- [flickr] update default directories and improve metadata consistency (#828)
- [sankaku] use API endpoints from
beta.sankakucomplex.com
- [downloader:http] improve filename extension handling (#776)
- replace all JPEG filename extensions with
jpg
by default
- [hentainexus] fix extraction (#1166)
- [instagram] rewrite (#1113, #1122, #1128, #1130, #1149)
- [mangadex] handle external chapters (#1154)
- [nozomi] handle empty
date
fields (#1163) - [paheal] create directory for each post (#1147)
- [piczel] update API URLs
- [twitter] update image URL format (#1145)
- [twitter] improve
x-csrf-token
header handling (#1170) - [webtoons] update
ageGate
cookies
- [sankaku] remove login support
- [2chan] skip external links
- [hentainexus] fix extraction (#1125)
- [mangadex] switch to API v2 (#1129)
- [mangapanda] use http://
- [mangoxo] fix extraction
- [reddit] skip invalid gallery items (#1127)
- [sankakucomplex] extract videos and embeds (#308)
- [twitter] add support for lists (#1096)
- [postprocessor:metadata] accept string-lists for
content-format
(#1080) - implement
modules
andextension-map
options
- [500px] update query hashes
- [8kun] fix file URLs of older posts (#1101)
- [exhentai] update image URL parsing (#1094)
- [hentaifoundry] update
YII_CSRF_TOKEN
cookie handling (#1083) - [hentaifoundry] use scheme from input URLs (#1095)
- [mangoxo] fix metadata extraction
- [paheal] fix extraction (#1088)
- collect post processors from
basecategory
entries (#1084)
- [pinterest] implement login support (#1055)
- [reddit] add
date
metadata field (#1068) - [seiga] add metadata for single image downloads (#1063)
- [twitter] support media from Cards (#937, #1005)
- [weasyl] support api-key authentication (#1057)
- add a
t
format string conversion for trimming whitespace (#1065)
- [blogger] handle URLs with specified width/height (#1061)
- [fallenangels] fix extraction of
.5
chapters - [gelbooru] rewrite mp4 video URLs (#1048)
- [hitomi] fix image URLs and gallery URL pattern
- [mangadex] unescape more metadata fields (#1066)
- [mangahere] ensure download URLs have a scheme (#1070)
- [mangakakalot] ignore "Go Home" buttons in chapter pages
- [newgrounds] handle embeds without scheme (#1033)
- [newgrounds] provide fallback URLs for video downloads (#1042)
- [xhamster] fix user profile extraction
- [hentaicafe] add
manga_id
metadata field (#1036) - [hentaifoundry] add support for stories (#734)
- [hentaifoundry] add
include
option - [newgrounds] extract image embeds (#1033)
- [nijie] add
include
option (#1018) - [reactor] match URLs without subdomain (#1053)
- [twitter] extend
retweets
option (#1026) - [weasyl] add extractors (#977)
- [500px] update query hashes
- [behance] fix
collection
extraction - [newgrounds] fix video extraction (#1042)
- [twitter] improve twitpic extraction (#1019)
- [weibo] handle posts with more than 9 images (#926)
- [xvideos] fix
title
extraction - fix crash when using
--download-archive
with--no-skip
(#1023) - fix issues with
blacklist
/whitelist
defaults (#1051, #1056)
- [kissmanga] remove module
- [deviantart] support watchers-only/paid deviations (#995)
- [myhentaigallery] add gallery extractor (#1001)
- [twitter] support specifying users by ID (#980)
- [twitter] support
/intent/user?user_id=…
URLs (#980) - add
--no-skip
command-line option (#986) - add
blacklist
andwhitelist
options (#492, #844) - add
filesize-min
andfilesize-max
options (#780) - add
sleep-extractor
andsleep-request
options (#788) - write skipped files to archive (#550)
- [exhentai] update wait time before original image downloads (#978)
- [imgur] use new API endpoints for image/album data
- [tumblr] create directories for each post (#965)
- support format string replacement fields in download archive paths (#985)
- reduce wait time growth rate for HTTP retries from exponential to linear
- [500px] update query hash
- [aryion] improve post ID extraction (#981, #982)
- [danbooru] handle posts without
id
(#1004) - [furaffinity] update download URL extraction (#988)
- [imgur] fix image/album detection for galleries
- [postprocessor:zip] defer zip file creation (#968)
- [jaiminisbox] remove extractors
- [worldthree] remove extractors
- [aryion] add username/password support (#960)
- [exhentai] add ability to specify a custom image limit (#940)
- [furaffinity] add
search
extractor (#915) - [imgur] add
search
andtag
extractors (#934)
- [500px] fix extraction and update URL patterns (#956)
- [aryion] update folder mime type list (#945)
- [gelbooru] fix extraction without API
- [hentaihand] update to new site layout
- [hitomi] fix redirect processing
- [reddit] handle deleted galleries (#953)
- [reddit] improve gallery extraction (#955)
- [blogger] add
search
extractor (#925) - [blogger] support searching posts by labels (#925)
- [inkbunny] add
user
andpost
extractors (#283) - [instagram] support
/reel/
URLs - [pinterest] support
pinterest.co.uk
URLs (#914) - [reddit] support gallery posts (#920)
- [subscribestar] extract attached media files (#852)
- [blogger] improve error messages for missing posts/blogs (#903)
- [exhentai] adjust image limit costs (#940)
- [gfycat] skip malformed gfycat responses (#902)
- [imgur] handle 403 overcapacity responses (#910)
- [instagram] wait before GraphQL requests (#901)
- [mangareader] fix extraction
- [mangoxo] fix login
- [pixnet] detect password-protected albums (#177)
- [simplyhentai] fix
gallery_id
extraction - [subscribestar] update
date
parsing - [vsco] handle missing
description
fields - [xhamster] fix extraction (#917)
- allow
parent-directory
to work recursively (#905) - skip external OAuth tests (#908)
- [bobx] remove module
- [8muses] support
comics.8muses.com
URLs - [artstation] add
following
extractor (#888) - [exhentai] add
domain
option (#897) - [gfycat] add
user
andsearch
extractors - [imgur] support all
/t/...
URLs (#880) - [khinsider] add
format
option (#840) - [mangakakalot] add
manga
andchapter
extractors (#876) - [redgifs] support
gifsdeliverynetwork.com
URLs (#874) - [subscribestar] add
user
andpost
extractors (#852) - [twitter] add support for nitter.net URLs (#890)
- add Zsh completion script (#150)
- [gfycat] retry 404'ed videos on redgifs.com (#874)
- [newgrounds] fix favorites extraction
- [patreon] yield images and attachments before post files (#871)
- [reddit] fix AttributeError when using
recursion
(#879) - [twitter] raise proper exception if a user doesn't exist (#891)
- defer directory creation (#722)
- set pseudo extension for Metadata messages (#865)
- prevent exception on Cloudflare challenges (#868)
- [artstation] add
date
metadata field (#839) - [mastodon] add
date
metadata field (#839) - [pinterest] add support for board sections (#835)
- [twitter] add extractor for liked tweets (#837)
- [twitter] add option to filter media from quoted tweets (#854)
- [weibo] add
date
metadata field tostatus
objects (#829)
- [aryion] fix user gallery extraction (#832)
- [imgur] build directory paths for each file (#842)
- [tumblr] prevent errors when using
reblogs=same-blog
(#851) - [twitter] always provide an
author
metadata field (#831, #833) - [twitter] don't download video previews (#833)
- [twitter] improve handling of deleted tweets (#838)
- [twitter] fix search results (#847)
- [twitter] improve handling of quoted tweets (#854)
- fix config lookups when multiple locations are involved (#843)
- improve output of
-K/--list-keywords
for parent extractors (#825) - call
flush()
after writing JSON inDataJob()
(#727)
- [deviantart] extend
extra
option; also search journals for sta.sh links (#712) - [twitter] rewrite; use new interface (#806, #740)
- [kissmanga] work around CAPTCHAs (#818)
- [nhentai] fix extraction (#819)
- [webtoons] generalize comic extraction code (#820)
- [imagechest] add new extractor for imgchest.com (#750)
- [instagram] add
post_url
,tags
,location
,tagged_users
metadata (#743) - [redgifs] add image extractor (#724)
- [webtoons] add new extractor for webtoons.com (#761)
- implement
--write-pages
option (#736) - extend
path-restrict
option (#662) - implement
path-replace
option (#662, #755) - make
path
andkeywords
available in logging messages (#574, #575)
- [danbooru] change default value of
ugoira
tofalse
- [downloader:ytdl] change default value of
forward-cookies
tofalse
- [downloader:ytdl] fix file extensions when merging into
.mkv
(#720) - write OAuth tokens to cache (#616)
- use
%APPDATA%\gallery-dl
for config files and cache on Windows - use
util.Formatter
for formatting logging messages - reuse HTTP connections from parent extractors
- [deviantart] use private access tokens for Journals (#738)
- [gelbooru] simplify and fix pool extraction
- [imgur] fix extraction of animated images without
mp4
entry - [imgur] treat
/t/unmuted/
URLs as galleries - [instagram] fix login with username & password (#756, #771, #797, #803)
- [reddit] don't send OAuth headers for file downloads (#729)
- fix/improve Cloudflare bypass code (#728, #757)
- reset filenames on empty file extensions (#733)
- [patreon] respect filters and sort order in query parameters (#711)
- [speakerdeck] add a new extractor for speakerdeck.com (#726)
- [twitter] add
replies
option (#705) - [weibo] add
videos
option - [downloader:http] add MIME types for
.psd
files (#714)
- [artstation] improve embed extraction (#720)
- [deviantart] limit API wait times (#721)
- [newgrounds] fix URLs produced by the
following
extractor (#684) - [patreon] improve file hash extraction (#713)
- [vsco] fix user gallery extraction
- fix/improve Cloudflare bypass code (#728)
- [500px] recognize
web.500px.com
URLs - [aryion] support downloading from folders (#694)
- [furaffinity] add extractor for followed users (#515)
- [hitomi] add extractor for tag searches (#697)
- [instagram] add
post_id
andnum
metadata fields (#698) - [newgrounds] add extractor for followed users (#684)
- [patreon] recognize URLs with creator IDs (#711)
- [twitter] add
reply
metadata field (#705) - [xhamster] recognize
xhamster.porncache.net
URLs (#700)
- [gelbooru] improve post ID extraction in pool listings
- [hitomi] fix extraction of galleries without tags
- [jaiminisbox] update metadata decoding procedure (#702)
- [mastodon] fix pagination (#701)
- [mastodon] improve account searches (#704)
- [patreon] fix hash extraction from download URLs (#693)
- improve parameter extraction when solving Cloudflare challenges
- [aryion] add
gallery
andpost
extractors (#390, #673) - [deviantart] detect and handle folders in sta.sh listings (#659)
- [hentainexus] add
circle
,event
, andtitle_conventional
metadata fields (#661) - [hiperdex] add
artist
extractor (#606) - [mastodon] add access tokens for
mastodon.social
andbaraag.net
(#665)
- [deviantart] retrieve all download URLs through the OAuth API
- automatically read config files in PyInstaller executable directories (#682)
- [deviantart] handle "Request blocked" errors (#655)
- [deviantart] improve JPEG quality replacement pattern
- [hiperdex] fix extraction
- [mastodon] handle API rate limits (#665)
- [mastodon] update OAuth credentials for pawoo.net (#665)
- [myportfolio] fix extraction of galleries without title
- [piczel] fix extraction of single images
- [vsco] fix collection extraction
- [weibo] accept status URLs with non-numeric IDs (#664)
- [instagram] Add support for user's saved medias (#644)
- [nozomi] support multiple images per post (#646)
- [35photo] add
tag
extractor
- [mangadex] transform timestamps from
date
fields to datetime objects
- [deviantart] handle decode errors for
extended_fetch
results (#655) - [e621] fix bug in API rate limiting and improve pagination (#651)
- [instagram] update pattern for user profile URLs
- [mangapark] fix metadata extraction
- [nozomi] sort search results (#646)
- [piczel] fix extraction
- [twitter] fix typo in
x-twitter-auth-type
header (#625) - remove trailing dots from Windows directory names (#647)
- fix crash with missing
stdout
/stderr
/stdin
handles (#653)
- [furaffinity] extract more metadata
- [instagram] add
post_shortcode
metadata field (#525) - [kabeuchi] add extractor (#561)
- [newgrounds] add extractor for favorited posts (#394)
- [pixiv] implement
avatar
option (#595, #623) - [twitter] add extractor for bookmarked Tweets (#625)
- [bcy] reduce number of HTTP requests during data extraction
- [e621] update to new interface (#635)
- [exhentai] handle incomplete MIME types (#632)
- [hitomi] improve metadata extraction
- [mangoxo] fix login
- [newgrounds] improve error handling when extracting post data
- [hentaihand] add extractors (#605)
- [hiperdex] add chapter and manga extractors (#606)
- [oauth] implement option to write DeviantArt refresh-tokens to cache (#616)
- [downloader:http] add more MIME types for
.bmp
and.rar
files (#621, #628) - warn about expired cookies
- [bcy] fix partial image URLs (#613)
- [danbooru] fix Ugoira downloads and metadata
- [deviantart] check availability of
/intermediary/
URLs (#609) - [hitomi] follow multiple redirects & fix image URLs
- [piczel] improve and update
- [tumblr] replace
-
with - [vsco] update gallery URL pattern
- fix
--verbose
and--quiet
command-line options
- Support for
furaffinity
- https://www.furaffinity.net/ (#284)8kun
- https://8kun.top/ (#582)bcy
- https://bcy.net/ (#592)
- [blogger] implement video extraction (#587)
- [oauth] add option to specify port number used by local server (#604)
- [pixiv] add
rating
metadata field (#595) - [pixiv] recognize tags at the end of new bookmark URLs
- [reddit] add
videos
option - [weibo] use youtube-dl to download from m3u8 manifests
- implement
parent-directory
option (#551) - extend filename formatting capabilities:
- [reddit] handle reddit-hosted images and videos natively (#551)
- [twitter] change default value for
videos
totrue
- [cloudflare] unescape challenge URLs
- [deviantart] fix video extraction from
extended_fetch
results - [hitomi] implement workaround for "broken" redirects
- [khinsider] fix and improve metadata extraction
- [patreon] filter duplicate files per post (#590)
- [piczel] fix extraction
- [pixiv] fix user IDs for bookmarks API calls (#596)
- [sexcom] fix image URLs
- [twitter] force old login page layout (#584, #598)
- [vsco] skip "invalid" entities
- improve functions to load/save cookies.txt files (#586)
- [yaplog] remove module
- [hentaifoundry] extract more metadata (#565)
- [twitter] add option to extract TwitPic embeds (#579)
- implement a post-processor module to compare file versions (#530)
- [hitomi] update image URL generation
- [mangadex] revert domain to
mangadex.org
- [pinterest] improve detection of invalid pin.it links
- [pixiv] update URL patterns for user profiles and bookmarks (#568)
- [twitter] Fix stop before real end (#573)
- remove temp files before downloading from fallback URLs
- [erolord] remove extractor
- [deviantart] match new search/popular URLs (#538)
- [deviantart] match
/favourites/all
URLs (#555) - [deviantart] add extractor for followed users (#515)
- [pixiv] support listing followed users (#515)
- [imagefap] handle beta.imagefap.com URLs (#552)
- [postprocessor:metadata] add
directory
option (#520)
- [artstation] fix search result pagination (#537)
- [directlink] send Referer headers (#536)
- [exhentai] restrict default directory name length (#545)
- [mangadex] change domain to mangadex.cc (#559)
- [mangahere] send
isAdult
cookies (#556) - [newgrounds] fix tags metadata extraction
- [pixiv] retry after rate limit errors (#535)
- [twitter] handle quoted tweets (#526)
- [twitter] handle API rate limits (#526)
- [twitter] fix URLs forwarded to youtube-dl (#540)
- prevent infinite recursion when spawning new extractors (#489)
- improve output of
--list-keywords
for "parent" extractors (#548) - provide fallback for SQLite versions with missing
WITHOUT ROWID
support (#553)
- [4chan] add extractor for entire boards (#510)
- [realbooru] add extractors for pools, posts, and tag searches (#514)
- [instagram] implement a
videos
option (#521) - [vsco] implement a
videos
option - [postprocessor:metadata] implement a
bypost
option for downloading the metadata of an entire post (#511)
- [reddit] change the default value for
comments
to0
- [vsco] improve image resolutions
- make filesystem-related errors during file downloads non-fatal (#512)
- [foolslide] add fallback for chapter data extraction
- [instagram] ignore errors during post-page extraction
- [patreon] avoid errors when fetching user info (#508)
- [patreon] improve URL pattern for single posts
- [reddit] fix errors with
t1
submissions - [vsco] fix user profile extraction … again
- [weibo] handle unavailable/deleted statuses
- [downloader:http] improve rate limit handling
- retain trailing zeroes in Cloudflare challenge answers
- [flickr] support 3k, 4k, 5k, and 6k photo sizes (#472)
- [imgur] add extractor for subreddit links (#500)
- [newgrounds] add extractors for
audio
listings and generalmedia
files (#394) - [newgrounds] implement login support (#394)
- [postprocessor:metadata] implement a
extension-format
option (#477) --exec-after
- [deviantart] ensure consistent username capitalization (#455)
- [directlink] split
{path}
into{path}/{filename}.{extension}
- [twitter] update metadata fields with user/author information
- [postprocessor:metadata] filter private entries & rename
format
tocontent-format
- Enable
cookies-update
by default
- [2chan] fix metadata extraction
- [behance] get images from 'media_collection' modules
- [bobx] fix image downloads by randomly generating session cookies (#482)
- [deviantart] revert to getting download URLs from OAuth API calls (#488)
- [deviantart] fix URL generation from '/extended_fetch' results (#505)
- [flickr] adjust OAuth redirect URI (#503)
- [hentaifox] fix extraction
- [imagefap] adapt to new image URL format
- [imgbb] fix error in galleries without user info (#471)
- [instagram] prevent errors with missing 'video_url' fields (#479)
- [nijie] fix
date
parsing - [pixiv] match new search URLs (#507)
- [plurk] fix comment pagination
- [sexcom] send specific Referer headers when downloading videos
- [twitter] fix infinite loops (#499)
- [vsco] fix user profile and collection extraction (#480)
- Fix Cloudflare DDoS protection bypass
--abort-on-skip
- Fix inclusion of bash completion and man pages in source distributions
- Support for
blogger
- https://www.blogger.com/ (#364)nozomi
- https://nozomi.la/ (#388)issuu
- https://issuu.com/ (#413)naver
- https://blog.naver.com/ (#447)
- Extractor for
twitter
search results (#448) - Extractor for
deviantart
user profiles with configurable targets (#377, #419) --ugoira-conv-lossless
(#432)cookies-update
option to allow updating cookies.txt files (#445)- Optional
cloudflare
andvideo
installation targets (#460) - Allow executing commands with the
exec
post-processor after all files are downloaded (#413, #421)
- Rewrite
imgur
using its public API (#446) - Rewrite
luscious
using GraphQL queries (#457) - Adjust default
nijie
filenames to matchpixiv
- Change enumeration index for gallery extractors from
page
tonum
- Return non-zero exit status when errors occurred
- Forward proxy settings to youtube-dl downloader
- Install bash completion script into
share/bash-completion/completions
- Adapt to new
instagram
page layout when logged in (#391) - Support protected
twitter
videos (#452) - Extend
hitomi
URL pattern and fix gallery extraction - Restore OAuth2 authentication error messages
- Miscellaneous fixes for
patreon
(#444),deviantart
(#455),sexcom
(#464),imgur
(#467),simplyhentai
--exec
command-line option to specify a command to run after each file download (#421)
- Include titles in
gfycat
default filenames (#434)
- Fetch working download URLs for
deviantart
(#436) - Various fixes and improvements for
yaplog
blogs (#443) - Fix image URL generation for
hitomi
galleries - Miscellaneous fixes for
behance
andxvideos
instagram.highlights
option to include highlighted stories when downloading user profiles (#329)- Support for
/user/
URLs onreddit
(#350) - Support for
imgur
user profiles and favorites (#420) - Additional metadata fields on
nijie
(#423)
- Improve handling of private
deviantart
artworks (#414) and 429 status codes (#424) - Prevent fatal errors when trying to open download-archive files (#417)
- Detect and ignore unavailable videos on
weibo
(#427) - Update the
scope
of newreddit
refresh-tokens (#428) - Fix inconsistencies with the
reddit.comments
option (#429) - Extend URL patterns for
hentaicafe
manga andpixiv
artworks - Improve detection of unavailable albums on
luscious
andimgbb
- Miscellaneous fixes for
tsumino
- Support for
lineblog
- https://www.lineblog.me/ (#404)fuskator
- https://fuskator.com/ (#407)
ugoira
option fordanbooru
to download pre-rendered ugoira animations (#406)
- Download the correct files from
twitter
replies (#403) - Prevent crash when trying to use unavailable downloader modules (#405)
- Fix
pixiv
authentication (#411) - Improve
exhentai
image limit checks - Miscellaneous fixes for
hentaicafe
,simplyhentai
,tumblr
- Provide
filename
metadata for alldeviantart
files (#392, #400) - Implement a
ytdl.outtmpl
option to let youtube-dl handle filenames by itself (#395) - Support
seiga
mobile URLs (#401)
- Extract more than the first 32 posts from
piczel
galleries (#396) - Fix filenames of archives created with
--zip
(#397) - Skip unavailable images and videos on
flickr
(#398) - Fix filesystem paths on Windows with Python 3.6 and lower (#402)
- Support for
instagram
stories and IGTV (#371, #373) - Support for individual
imgbb
images (#363) deviantart.quality
option to set the JPEG compression quality for newer images (#369)enumerate
option forextractor.skip
(#306)adjust-extensions
option to control filename extension adjustmentspath-remove
option to remove control characters etc. from filesystem paths
- Rename
restrict-filenames
topath-restrict
- Adjust
pixiv
metadata and default filename format (#366)- Set
filename
to"{category}_{user[id]}_{id}{suffix}.{extension}"
to restore the old default
- Set
- Improve and optimize directory and filename generation
- Allow the
classify
post-processor to handle files with unknown filename extension (#138) - Fix rate limit handling for OAuth APIs (#368)
- Fix artwork and scraps extraction on
deviantart
(#376, #392) - Distinguish between
imgur
album and gallery URLs (#380) - Prevent crash when using
--ugoira-conv
(#382) - Handle multi-image posts on
patreon
(#383) - Miscellaneous fixes for
*reactor
,simplyhentai
- Use the correct domain for exhentai.org input URLs
- Prior to version 1.10.0 all cache files were created world readable (mode
644
) leading to possible sensitive information disclosure on multi-user systems - It is recommended to restrict access permissions of already existing files
(
/tmp/.gallery-dl.cache
) withchmod 600
- Windows users should not be affected
- Support for
vsco
- https://vsco.co/ (#331)imgbb
- https://imgbb.com/ (#361)adultempire
- https://www.adultempire.com/ (#340)
restrict-filenames
option to create Windows-compatible filenames on any platform (#348)forward-cookies
option to control cookie forwarding to youtube-dl (#352)
- The default cache file location on non-Windows systems is now
$XDG_CACHE_HOME/gallery-dl/cache.sqlite3
or~/.cache/gallery-dl/cache.sqlite3
- New cache files are created with mode
600
exhentai
extractors will always usee-hentai.org
as domain
- Better handling of
exhentai
image limits and errors (#356, #360) - Try to prevent ZIP file corruption (#355)
- Miscellaneous fixes for
behance
,ngomik
- Support for
erolord
- http://erolord.com/ (#326)
- Add login support for
instagram
(#195) - Add
--no-download
andextractor.*.download
disable file downloads (#220) - Add
-A/--abort
to specify the number of consecutive download skips before aborting - Interpret
-1
as infinite retries (#300) - Implement custom log message formats per log-level (#304)
- Implement an
mtime
post-processor that sets file modification times according to metadata fields (#332) - Implement a
twitter.content
option to enable tweet text extraction (#333, #338) - Enable
date-min/-max/-format
options fortumblr
(#337)
- Set file modification times according to their
Last-Modified
header when downloading (#236, #277)- Use
--no-mtime
ordownloader.*.mtime
to disable this behavior
- Use
- Duplicate download URLs are no longer silently ignored (controllable with
extractor.*.image-unique
) - Deprecate
--abort-on-skip
- Retry downloads on OpenSSL exceptions (#324)
- Ignore unavailable pins on
sexcom
instead of raising an exception (#325) - Use Firefox's SSL/TLS ciphers to prevent Cloudflare CAPTCHAs (#342)
- Improve folder name matching on
deviantart
(#343) - Forward cookies to
youtube-dl
to allow downloading private videos - Miscellaneous fixes for
35photo
,500px
,newgrounds
,simplyhentai
- Support for
vanillarock
- https://vanilla-rock.com/ (#254)nsfwalbum
- https://nsfwalbum.com/ (#287)
artist
andtags
metadata forhentaicafe
(#238)description
metadata forinstagram
(#310)- Format string option to replace a substring with another -
R<old>/<new>/
(#318)
- Delete empty archives created by the
zip
post-processor (#316)
- Handle
hitomi
Game CG galleries correctly (#321) - Miscellaneous fixes for
deviantart
,hitomi
,pururin
,kissmanga
,keenspot
,mangoxo
,imagefap
- Support for
slickpic
- https://www.slickpic.com/ (#249)xhamster
- https://xhamster.com/ (#281)pornhub
- https://www.pornhub.com/ (#282)8muses
- https://www.8muses.com/ (#305)
extra
option fordeviantart
to download Sta.sh content linked in description texts (#302)
- Detect
directlink
URLs with upper case filename extensions (#296)
- Improved error handling for
tumblr
API calls (#297) - Fixed extraction of
livedoor
blogs (#301) - Fixed extraction of special
deviantart
Sta.sh items (#307) - Fixed pagination for specific
keenspot
comics
- Support for
keenspot
- http://keenspot.com/ (#223)sankakucomplex
- https://www.sankakucomplex.com (#258)
folders
option fordeviantart
to add a list of containing folders to each file (#276)captcha
option forkissmanga
andreadcomiconline
to control CAPTCHA handling (#279)filename
metadata for files downloaded with youtube-dl (#291)
- Adjust
wallhaven
extractors to new page layout:- use API and add
api-key
option - removed traditional login support
- use API and add
- Provide original filenames for
patreon
downloads (#268) - Use e-hentai.org or exhentai.org depending on input URL (#278)
- Fix pagination over
sankaku
popular listings (#265) - Fix folder and collection extraction on
deviantart
(#271) - Detect "AreYouHuman" redirects on
readcomiconline
(#279) - Miscellaneous fixes for
hentainexus
,livedoor
,ngomik
- Support for
patreon
- https://www.patreon.com/ (#226)hentainexus
- https://hentainexus.com/ (#256)
date
metadata fields forpixiv
(#248),instagram
(#250),exhentai
, andnewgrounds
- Improved
flickr
metadata and video extraction (#246)
- Download original GIF animations from
deviantart
(#242) - Ignore missing
edge_media_to_comment
fields oninstagram
(#250) - Fix serialization of
datetime
objects for--write-metadata
(#251, #252) - Allow multiple post-processor command-line options at once (#253)
- Prevent crash on
booru
sites when no tags are available (#259) - Fix extraction on
instagram
afterrhx_gis
field removal (#266) - Avoid Cloudflare CAPTCHAs for Python interpreters built against OpenSSL < 1.1.1
- Miscellaneous fixes for
luscious
- Support for
plurk
- https://www.plurk.com/ (#212)sexcom
- https://www.sex.com/ (#147)
--clear-cache
date
metadata fields fordeviantart
,twitter
, andtumblr
(#224, #232)
- Standalone executables are now built using PyInstaller:
- uses the latest CPython interpreter (Python 3.7.3)
- available on several platforms (Windows, Linux, macOS)
- includes the
certifi
CA bundle,youtube-dl
, andpyOpenSSL
on Windows
- Patch
urllib3
's default list of SSL/TLS ciphers to prevent Cloudflare CAPTCHAs (#227) (Windows users need to installpyOpenSSL
for this to take effect) - Provide fallback URLs for
twitter
images (#237) - Send
Referer
headers when downloading fromhitomi
(#239) - Updated login procedure on
mangoxo
- Support for
pixnet
- https://www.pixnet.net/ (#177)wikiart
- https://www.wikiart.org/ (#179)mangoxo
- https://www.mangoxo.com/ (#184)yaplog
- https://yaplog.jp/ (#190)livedoor
- http://blog.livedoor.jp/ (#190)
- Login support for
mangoxo
(#184) andtwitter
(#214)
- Increased required
Requests
version to 2.11.0
- Improved image quality on
reactor
sites (#210) - Support
imagebam
galleries with more than 100 images (#219) - Updated Cloudflare bypass code
- Support for:
35photo
- https://35photo.pro/ (#162)500px
- https://500px.com/ (#185)
instagram
extractor for hashtags (#202)- Option to get more metadata on
deviantart
(#189) - Man pages and bash completion (#150)
- Snap improvements (#197, #199, #207)
- Better FFmpeg arguments for
--ugoira-conv
- Adjusted metadata for
luscious
albums
- Proper handling of
instagram
multi-image posts (#178, #201) - Fixed
tumblr
avatar URLs when not using OAuth1.0 (#193) - Miscellaneous fixes for
exhentai
,komikcast
- Support for:
weibo
- https://www.weibo.com/pururin
- https://pururin.io/ (#174)fashionnova
- https://www.fashionnova.com/ (#175)shopify
sites in general (#175)
- Snap packaging (#169, #170, #187, #188)
- Automatic Cloudflare DDoS protection bypass
- Extractor and Job information for logging format strings
dynastyscans
image and search extractors (#163)deviantart
scraps extractor (#168)artstation
extractor for artwork listings (#172)smugmug
video support and improved image format selection (#183)
- More metadata for
nhentai
galleries - Combined
myportfolio
extractors into one - Renamed
name
metadata field tofilename
and removed the originalfilename
field - Simplified and improved internal data structures
- Optimized creation of child extractors
- Filter empty
tumblr
URLs (#165) - Filter ads and improve connection speed on
hentaifoundry
- Show proper error messages if
luscious
galleries are unavailable - Miscellaneous fixes for
mangahere
,ngomik
,simplyhentai
,imgspice
seaotterscans
- Added support for:
photobucket
- http://photobucket.com/ (#117)hentaifox
- https://hentaifox.com/ (#160)tsumino
- https://www.tsumino.com/ (#161)
- Added the ability to dynamically generate extractors based on a user's config file for
- Added an extractor for
behance
collections (#157) - Added login support for
luscious
(#159) andtsumino
(#161) - Added an option to stop downloading if the
exhentai
image limit is exceeded (#141) - Fixed extraction issues for
behance
andmangapark
- Added
metadata
post-processor to write image metadata to an external file (#135) - Added option to reverse chapter order of manga extractors (#149)
- Added authentication support for
danbooru
(#151) - Added tag metadata for
exhentai
andhbrowse
galleries - Improved
*reactor
extractors (#148) - Fixed extraction issues for
nhentai
(#156),pinterest
,mangapark
- Added support for:
instagram
- https://www.instagram.com/ (#134)
- Added support for multiple items on sta.sh pages (#113)
- Added option to download
tumblr
avatars (#137) - Changed defaults for visited post types and inline media on
tumblr
- Improved inline extraction of
tumblr
posts (#133, #137) - Improved error handling and retry behavior of all API calls
- Improved handling of missing fields in format strings (#136)
- Fixed hash extraction for unusual
tumblr
URLs (#129) - Fixed image subdomains for
hitomi
galleries (#142) - Fixed and improved miscellaneous issues for
kissmanga
(#20),luscious
,mangapark
,readcomiconline
- Added support for:
joyreactor
- http://joyreactor.cc/ (#114)pornreactor
- http://pornreactor.cc/ (#114)newgrounds
- https://www.newgrounds.com/ (#119)
- Added extractor for search results on
luscious
(#127) - Fixed filenames of ZIP archives (#126)
- Fixed extraction issues for
gfycat
,hentaifoundry
(#125),mangafox
- Added support for:
wallhaven
- https://alpha.wallhaven.cc/yuki
- https://yuki.la/
- Added youtube-dl integration and video downloads for
twitter
(#99),behance
,artstation
- Added per-extractor options for network connections (
retries
,timeout
,verify
) - Added a
--no-check-certificate
command-line option - Added ability to specify the number of skipped downloads before aborting/exiting (#115)
- Added extractors for scraps, favorites, popular and recent images on
hentaifoundry
(#110) - Improved login procedure for
pixiv
to avoid unwanted emails on each new login - Improved album metadata and error handling for
flickr
(#109) - Updated default User-Agent string to Firefox 62 (#122)
- Fixed
twitter
API response handling when logged in (#123) - Fixed issue when converting Ugoira using H.264
- Fixed miscellaneous issues for
2chan
,deviantart
,fallenangels
,flickr
,imagefap
,pinterest
,turboimagehost
,warosu
,yuki
(#112)
- Added support for:
hentaicafe
- https://hentai.cafe/ (#101)bobx
- http://www.bobx.com/dark/
- Added black-/whitelist options for post-processor modules
- Added support for
tumblr
inline videos (#102) - Fixed extraction of
smugmug
albums without owner (#100) - Fixed issues when using default config values with
reddit
extractors (#104) - Fixed pagination for user favorites on
sankaku
(#106) - Fixed a crash when processing
deviantart
journals (#108)
- Added support for
twitter
timelines (#96) - Added option to suppress FFmpeg output during ugoira conversions
- Improved filename formatter performance
- Improved inline image quality on
tumblr
(#98) - Fixed image URLs for newly released
mangadex
chapters - Fixed a smaller issue with
deviantart
journals - Replaced
subapics
withngomik
- Added support for:
piczel
- https://piczel.tv/
- Added support for related pins on
pinterest
- Fixed accessing "offensive" galleries on
exhentai
(#97) - Fixed extraction issues for
mangadex
,komikcast
andbehance
- Removed original-image functionality from
tumblr
, since "raw" images are no longer accessible
- Added support for:
behance
- https://www.behance.net/myportfolio
- https://www.myportfolio.com/ (#95)
- Added custom format string options to handle long strings (#92, #94)
- Slicing:
"{field[10:40]}"
- Replacement:
"{field:L40/too long/}"
- Slicing:
- Improved frame rate handling for ugoira conversions
- Improved private access token usage on
deviantart
- Fixed metadata extraction for some images on
nijie
- Fixed chapter extraction on
mangahere
- Removed
whatisthisimnotgoodwithcomputers
- Removed support for Python 3.3
- Added image-pool extractors for
safebooru
andrule34
- Added option for extended tag information on
booru
sites (#92) - Added support for DeviantArt's new URL format
- Added support for
mangapark
mirrors - Changed
imagefap
extractors to use HTTPS - Fixed crash when skipping downloads for files without known extension
- Added an
ugoira
post-processor to convertpixiv
animations to WebM - Added
--zip
and--ugoira-conv
command-line options - Changed how ugoira frame information is handled
- instead of being written to a separate file, it is now made available as metadata field of the ZIP archive
- Fixed manga and chapter titles for
mangadex
- Fixed file deletion by post-processors
- Added support for:
simplyhentai
- https://www.simply-hentai.com/ (#89)
- Added extractors for
pixiv
search results and followed usersdeviantart
search results and popular listings
- Added post-processors to perform actions on downloaded files
- Added options to configure logging behavior
- Added OAuth support for
smugmug
- Changed
pixiv
extractors to use the AppAPI- this breaks
favorite
archive IDs and changes some metadata fields
- this breaks
- Changed the default filename format for
tumblr
and renamedoffset
tonum
- Fixed a possible UnicodeDecodeError during installation (#86)
- Fixed extraction of
mangadex
manga with more than 100 chapters (#84) - Fixed miscellaneous issues for
imgur
,reddit
,komikcast
,mangafox
andimagebam
- Added support for:
smugmug
- https://www.smugmug.com/
- Added title information for
mangadex
chapters - Improved the
pinterest
API implementation (#83) - Improved error handling for
deviantart
andtumblr
- Removed
gomanga
andpuremashiro
- Added support for custom OAuth2 credentials for
pinterest
- Improved rate limit handling for
tumblr
extractors - Improved
hentaifoundry
extractors - Improved
imgur
URL patterns - Fixed miscellaneous extraction issues for
luscious
andkomikcast
- Removed
loveisover
andspectrumnexus
- Added extractors for
nhentai
search resultsexhentai
search results and favoritesnijie
doujins and favorites
- Improved metadata extraction for
exhentai
andnijie
- Improved
tumblr
extractors by avoiding unnecessary API calls - Fixed Cloudflare DDoS protection bypass
- Fixed errors when trying to print unencodable characters
- Added extractors for
artstation
albums, challenges and search results - Improved URL and metadata extraction for
hitomi
andnhentai
- Fixed page transitions for
danbooru
API results (#82)
- Added support for:
mangadex
- https://mangadex.org/artstation
- https://www.artstation.com/
- Added Cloudflare DDoS protection bypass to
komikcast
extractors - Changed archive ID formats for
deviantart
folders and collections - Improved error handling for
deviantart
API calls - Removed
imgchili
and various smaller image hosts
- Added
--proxy
to explicitly specify a proxy server (#76) - Added options to customize archive ID formats and undefined replacement fields
- Changed various archive ID formats to improve their behavior for favorites / bookmarks / etc.
- Affected modules are
deviantart
,flickr
,tumblr
,pixiv
and all …boorus
- Affected modules are
- Improved
sankaku
andidolcomplex
support by- respecting
page
andnext
URL parameters (#79) - bypassing the page-limit for unauthenticated users
- respecting
- Improved
directlink
metadata by properly unquoting it - Fixed
pixiv
ugoira extraction (#78) - Fixed miscellaneous extraction issues for
mangastream
andtumblr
- Removed
yeet
,chronos
,coreimg
,hosturimage
,imageontime
,img4ever
,imgmaid
,imgupload
- Added support for:
paheal
- https://rule34.paheal.net/ (#69)komikcast
- https://komikcast.com/ (#70)subapics
- http://subapics.com/ (#70)
- Added
--download-archive
to record downloaded files in an archive file - Added
--write-log
to write logging output to a file - Added a filetype check on download completion to fix incorrectly assigned filename extensions (#63)
- Added the
tumblr:...
pseudo URI scheme to support custom domains for Tumblr blogs (#71) - Added fallback URLs for
tumblr
images (#64) - Added support for
reddit
-hosted images (#68) - Improved the input file format by allowing comments and per-URL options
- Fixed OAuth 1.0 signature generation for Python 3.3 and 3.4 (#75)
- Fixed smaller issues for
luscious
,hentai2read
,hentaihere
andimgur
- Removed the
batoto
module
- Added support for:
puremashiro
- http://reader.puremashiro.moe/ (#66)idolcomplex
- https://idol.sankakucomplex.com/
- Added an option to filter reblogs on
tumblr
(#61) - Added OAuth user authentication for
tumblr
(#65) - Added support for
slideshare
mobile URLs (#67) - Improved pagination for various …booru sites to work around page limits
- Fixed chapter information parsing for certain manga on
kissmanga
(#58) andbatoto
(#60)
- Added support for:
slideshare
- https://www.slideshare.net/ (#54)
- Added pool- and post-extractors for
sankaku
- Added OAuth user authentication for
deviantart
- Updated
luscious
to supportmembers.luscious.net
URLs (#55) - Updated
mangahere
to use their new domain name (mangahere.cc) and support mobile URLs - Updated
gelbooru
to not be restricted to the first 20,000 images (#56) - Fixed extraction issues for
nhentai
andkhinsider
- Added the
-r/--limit-rate
command-line option to set a maximum download rate - Added the
--sleep
command-line option to specify the number of seconds to sleep before each download - Updated
gelbooru
to no longer use their now disabled API - Fixed SWF extraction for
sankaku
(#52) - Fixed extraction issues for
hentai2read
andkhinsider
- Removed the deprecated
--images
and--chapters
options - Removed the
mangazuki
module
- Added an option to set a custom user-agent string
- Improved retry behavior for failed HTTP requests
- Improved
seiga
by providing better metadata and getting more than the latest 200 images - Improved
tumblr
by adding support for all post types, scanning for inline images and following external links (#48) - Fixed extraction issues for
hbrowse
,khinsider
andsenmanga
- Added support for:
xvideos
- https://www.xvideos.com/ (#45)
- Fixed exception handling during file downloads which could lead to a premature exit
- Fixed an issue with
tumblr
where not all images would be downloaded when using tags (#48) - Fixed extraction issues for
imgbox
(#47),mangastream
(#49) andmangahere
- Added support for:
warosu
- https://warosu.org/b4k
- https://arch.b4k.co/
- Added support for
pixiv
ranking lists - Added support for
booru
popular lists (danbooru
,e621
,konachan
,yandere
,3dbooru
) - Added the
--cookies
command-line andcookies
config option to load additional cookies - Added the
--filter
and--chapter-filter
command-line options to select individual images or manga-chapters by their metadata using simple Python expressions (#43) - Added the
verify
config option to control certificate verification during file downloads - Added config options to overwrite internally used API credentials (API Tokens & IDs)
- Added
-K
as a shortcut for--list-keywords
- Changed the
--images
and--chapters
command-line options to--range
and--chapter-range
- Changed keyword names for various modules to make them accessible by
--filter
. In general minus signs have been replaced with underscores (e.g.gallery-id
->gallery_id
). - Changed default filename formats for manga extractors to optionally use volume and title information
- Improved the downloader modules to use
.part
files and support resuming incomplete downloads (#29) - Improved
deviantart
by distinguishing between users and groups (#26), always using HTTPS, and always downloading full-sized original images - Improved
sankaku
by adding authentication support and fixing various other issues (#44) - Improved URL pattern for direct image links (#30)
- Fixed an issue with
luscious
not getting original image URLs (#33) - Fixed various smaller issues for
batoto
,hentai2read
(#38),jaiminisbox
,khinsider
,kissmanga
(#28, #46),mangahere
,pawoo
,twitter
- Removed
kisscomic
andyonkouprod
modules
- Added support for:
2chan
- https://www.2chan.net/4plebs
- https://archive.4plebs.org/archivedmoe
- https://archived.moe/archiveofsins
- https://archiveofsins.com/desuarchive
- https://desuarchive.org/fireden
- https://boards.fireden.net/loveisover
- https://archive.loveisover.me/nyafuu
- https://archive.nyafuu.org/rbt
- https://rbt.asia/thebarchive
- https://thebarchive.com/mangazuki
- https://mangazuki.co/
- Improved
reddit
to allow submission filtering by ID and human-readable dates - Improved
deviantart
to support group galleries and gallery folders (#26) - Changed
deviantart
to use better default path formats - Fixed extraction of larger
imgur
albums - Fixed some smaller issues for
pixiv
,batoto
andfallenangels
- Added support for:
reddit
- https://www.reddit.com/ (#15)flickr
- https://www.flickr.com/ (#16)gfycat
- https://gfycat.com/
- Added support for direct image links
- Added user authentication via OAuth for
reddit
andflickr
- Added support for user authentication data from
.netrc
files (#22) - Added a simple progress indicator for multiple URLs (#19)
- Added the
--write-unsupported
command-line option to write unsupported URLs to a file - Added documentation for all available config options (configuration.rst)
- Improved
pixiv
to support tags for user downloads (#17) - Improved
pixiv
to support shortened and http://pixiv.me/... URLs (#23) - Improved
imgur
to properly handle.gifv
images and provide better metadata - Fixed an issue with
kissmanga
where metadata parsing for some series failed (#20) - Fixed an issue with getting filename extensions from
Content-Type
response headers
- Added the
--abort-on-skip
option to stop extraction if a download would be skipped - Improved the output format of the
--list-keywords
option - Updated
deviantart
to support all media types and journals - Updated
fallenangels
to support their Vietnamese version - Fixed an issue with multiple tags on ...booru sites
- Removed the
yomanga
module
- Added support for https://pawoo.net/
- Added manga extractors for all FoOlSlide-based modules
- Added the
-q/--quiet
and-v/--verbose
options to control output verbosity - Added the
-j/--dump-json
option to dump extractor results in JSON format - Added the
--ignore-config
option - Updated the
exhentai
extractor to fall back to using the e-hentai version if no username is given - Updated
deviantart
to support sta.sh URLs - Fixed an issue with
kissmanga
which prevented image URLs from being decrypted properly (again) - Fixed an issue with
pixhost
where for an image inside an album it would always download the first image of that album (#13) - Removed the
mangashare
andreadcomics
modules
- Fixed an issue in
kissmanga
which prevented image URLs from being decrypted properly
- Added new extractors:
kireicake
- https://reader.kireicake.com/seaotterscans
- https://reader.seaotterscans.com/
- Added a favourites extractor for
deviantart
- Re-enabled the
kissmanga
module - Updated
nijie
to support multi-page image listings - Updated
mangastream
to support readms.net URLs - Updated
exhentai
to support e-hentai.org URLs - Updated
fallenangels
to support their new domain and site layout
- Added logging support
- Added the
-R/--retries
option to specify how often a download should be retried before giving up - Added the
--http-timeout
option to set a timeout for HTTP connections - Improved error handling/tolerance during HTTP file downloads (#10)
- Improved option parsing and the help message from
-h/--help
- Changed the way configuration values are used by prioritizing top-level values
- This allows for cmdline options like
-u/--username
to overwrite values set in configuration files
- This allows for cmdline options like
- Fixed an issue with
imagefap.com
where incorrectly reported gallery sizes would cause the extractor to fail (#9) - Fixed an issue with
seiga.nicovideo.jp
where invalid characters in an API response caused the XML parser to fail - Fixed an issue with
seiga.nicovideo.jp
where the filename extension for the first image would be used for all others - Removed support for old configuration paths on Windows
- Removed several modules:
mangamint
: site is downwhentai
: now requires account with VIP status for original imageskissmanga
: encrypted image URLs (will be re-added later)
- Added
--images
and--chapters
options- Specifies which images (or chapters) to download through a comma-separated list of indices or index-ranges
- Example:
--images -2,4,6-8,10-
will select images with index 1, 2, 4, 6, 7, 8 and 10 up to the last one
- Changed the
-g
/--get-urls
option- The amount of how often the -g option is given now determines up until which level URLs are resolved.
- See 3bca86618505c21628cd9c7179ce933a78d00ca2
- Changed several option keys:
directory_fmt
->directory
filename_fmt
->filename
download-original
->original
- Improved FoOlSlide-based extractors
- Fixed URL extraction for hentai2read
- Fixed an issue with deviantart, where the API access token wouldn't get refreshed
- Added new extractors:
- fallenangels (famatg.com)
- Fixed url- and data-extraction for:
- nhentai
- mangamint
- imagetwist
- Disabled InsecureConnectionWarning when no certificates are available
- Added new extractors:
- gomanga
- yomanga
- mangafox
- Fixed deviantart extractor failing - switched to using their API
- Fixed an issue with SQLite on Python 3.6
- Automated test builds via Travis CI
- Standalone executables for Windows
- Added new extractors:
- kisscomic
- readcomics
- yonkouprod
- jaiminisbox
- Added manga extractor to batoto-module
- Added user extractor to seiga-module
- Added
-i
/--input-file
argument to allow local files and stdin as input (like wget) - Added basic support for
file://
URLs- this allows for the recursive extractor to be applied to local files:
$ gallery-dl r:file://[path to file]
- Added a utility extractor to run unit test URLs
- Updated luscious to deal with API changes
- Fixed twitter to provide the original image URL
- Minor fixes to hentaifoundry
- Removed imgclick extractor
- Added new extractors:
- whentai
- readcomiconline
- sensescans, worldthree
- imgmaid, imagevenue, img4ever, imgspot, imgtrial, pixhost
- Added base class for extractors of FoOlSlide-based sites
- Changed default paths for configuration files on Windows
- old paths are still supported, but that will change in future versions
- Fixed aborting downloads if a single one failed (#5)
- Fixed cloudflare-bypass cache containing outdated cookies
- Fixed image URLs for hitomi and 8chan
- Updated deviantart to always provide the highest quality image
- Updated README.rst
- Removed doujinmode extractor
- Added new extractors:
- hentaihere
- dokireader
- rapidimg, picmaniac
- Added support to find filename extensions by Content-Type response header
- Fixed filename/path issues on Windows (#4):
- Enable path names with more than 260 characters
- Remove trailing spaces in path segments
- Updated Job class to automatically set category/subcategory keywords
- Added new extractors:
- rule34
- dynastyscans
- imagebam, coreimg, imgcandy, imgtrex
- Added login capabilities for batoto
- Added
--version
cmdline argument to print the current program version and exit - Added
--list-extractors
cmdline argument to print names of all extractor classes together with descriptions and example URLs - Added proper error messages if an image/user does not exist
- Added unittests for every extractor
- Added new extractors:
- luscious
- doujinmode
- hentaibox
- seiga
- imagefap
- Changed error output to use stderr instead of stdout
- Fixed broken pipes causing an exception-dump by catching BrokenPipeErrors
- New modules (imagetwist, turboimagehost)
- Manga-extractors: Download entire manga and not just single chapters
- Generic extractor (provisional)
- Better and configurable console output
- Windows support