756 Commits

Author SHA1 Message Date
071f4f0911 Trigger rebuild 2024-03-11 11:41:32 +02:00
813c5236b3 [Grouple] Back to the old chapters parsing behavior with option 2024-03-11 09:27:04 +02:00
b7613606c0 [TuMangaOnline] Fix regexp 2024-03-09 14:03:09 +02:00
0aa4ea01f7 [Grouple] Multiple translations support 2024-03-09 12:10:04 +02:00
20685598e3 [TuMangaOnline] Fix pages parsing 2024-03-09 11:22:30 +02:00
b1fb1bdc6b Merge pull request #615 from NagaYZ/feature-search-by-number
Feature search by number
2024-03-09 11:04:13 +02:00
Naga
a80bdcc611 full title name 2024-03-08 21:18:52 +01:00
Naga
4571e3e001 gitignore 2024-03-08 02:31:59 +01:00
Naga
38d7d97167 added search by id 2024-03-08 02:29:59 +01:00
3913e95b53 Merge pull request #608 from nghduc97/fix-mangabox-parser-white-spaces
fix mangaboxparser white space coding in search query
2024-03-04 11:56:07 +02:00
954c926a9f Merge pull request #588 from NagaYZ/feature-webtoon-original
Feature webtoon original
2024-03-04 11:55:52 +02:00
nghduc97
0973444c47 fix mangaboxparser white space coding in search query 2024-03-03 11:45:10 +07:00
Naga
368b61c21a fixed suggestions 2024-02-28 18:13:51 +01:00
Naga
efc9b3502c fixed suggestions 2024-02-28 18:03:12 +01:00
Naga
2e86c480ec Update src/main/kotlin/org/koitharu/kotatsu/parsers/site/all/WebtoonsParser.kt
Co-authored-by: Koitharu <nvasya95@gmail.com>
2024-02-28 16:01:36 +01:00
Naga
39eeaacdcb fixup! trying to improve speed 2024-02-25 16:32:31 +01:00
Naga
ea45e91aa7 trying to improve speed 2024-02-25 14:47:06 +01:00
Naga
a65e41e796 fix 2024-02-25 14:24:14 +01:00
Naga
91648756c9 added more language 2024-02-25 14:16:24 +01:00
Naga
a225234800 fixed chapter date 2024-02-25 14:11:30 +01:00
Naga
de58333dc4 fixup! fixup! fixed offset 2024-02-25 13:12:09 +01:00
Naga
24a25e3dbf fixup! fixed offset 2024-02-25 13:00:31 +01:00
Naga
8e5ae51067 fixed offset 2024-02-25 12:50:49 +01:00
Naga
86c981ae8e fixup! trying to speed up loading 2024-02-25 12:33:11 +01:00
Naga
b716c35438 trying to speed up loading 2024-02-25 12:30:11 +01:00
Naga
9e984510d4 added caching for title list to speed up loading 2024-02-25 03:30:00 +01:00
Naga
3c08119ccc added caching for title list to speed up loading 2024-02-25 03:12:12 +01:00
Naga
ada1b7b54c fixed sort descending 2024-02-25 00:38:17 +01:00
Naga
a7ed4fbcc3 fix import 2024-02-24 23:33:10 +01:00
Naga
1330feaea5 added webtoon original english support 2024-02-24 23:32:37 +01:00
103f578c61 Merge pull request #579 from NagaYZ/master
Update domains name
2024-02-24 12:19:58 +02:00
deee80ae71 Merge pull request #581 from NagaYZ/fix-popular-page-url
Fix popular page url
2024-02-24 12:19:37 +02:00
Naga
54350040ad fixed popular page url 2024-02-22 20:32:21 +01:00
Naga
3472b8ba00 Update ZinManga.kt 2024-02-22 17:09:36 +01:00
Naga
f2dd9af66c Update MagusManga.kt
fixed domain name and listurl acces
2024-02-22 16:56:55 +01:00
Naga
da46f7967a fixed bunch of domain name 2024-02-22 16:56:25 +01:00
Naga
ed3d6c5f98 Replaced GloryScans/GloryScansX with LunarScans/LunarHentai
domain changed
2024-02-22 16:55:28 +01:00
Naga
c4ea825df9 Replaced LightScans with Luascans , domain name changed 2024-02-22 16:54:49 +01:00
Naga
4120e3946c Update RagnarokScanlation.kt
new domain: ragnarokscanlation.net
2024-02-22 16:54:09 +01:00
Naga
c84b6acff2 Update EmperorScan.kt
new domain : emperorscan.com
2024-02-22 16:53:58 +01:00
Naga
159110c032 Update ResetScans.kt
new domain: reset-scans.xyz
2024-02-22 16:53:46 +01:00
Naga
93e1b38da2 Update MangaEsp.kt
new domain : mangaesp.net
2024-02-22 16:53:28 +01:00
3ff028c4e9 Merge pull request #572 from geexuan/baozimh_details_fix
Baozimh: Fix for not getting all chapter details
2024-02-18 08:25:57 +02:00
geexuans
9b46963015 Baozimh: Fix for not getting all chapter details 2024-02-17 16:04:24 +08:00
09e1c14f37 Merge pull request #565 from sandeepjak2007/Issue##561
Issue #561 Leading to non existing page
2024-02-14 12:03:01 +02:00
Sandeep Karanam
58af01b90a Issue #561 Leading to non existing page
Issue #561 fixed
2024-02-14 09:23:38 +05:30
e338237e2a [NudeMoon] Fix pages loading 2024-02-12 16:02:27 +02:00
014ea5ef49 [Manganato] Update domain 2024-02-10 14:30:10 +02:00
4a8c7fa36a Fix urlDecode usage 2024-02-07 09:33:33 +02:00
1251a6cd75 Merge pull request #548 from Ah3ron/patch-1
Fix tag filtering for Russian characters
2024-02-07 09:11:57 +02:00
Ah3ron
0ce8eae969 Fix tag filtering for Russian characters 2024-02-05 19:45:35 +03:00
devi
8b9c76fca5 LerYaoi #512
StellarSaber #533
Update Url ( #530 )
Remove dead sources
2024-02-03 11:00:02 +01:00
52e4e6b802 Update collections library 2024-01-31 15:04:00 +02:00
57c9d26916 Fix android compatibility 2024-01-31 14:45:39 +02:00
19d7003a9e [BatoTo] Fix pages parsing #521 #522 2024-01-31 14:31:06 +02:00
b11e50ab22 Fixes batch 2024-01-30 13:09:27 +02:00
a03ab0e0c0 Update tests 2024-01-30 11:16:28 +02:00
05d4db00b3 [ResetScans] Fix chapters names #518 2024-01-30 11:05:41 +02:00
f4aaa14fc5 Simplify workflow 2024-01-30 10:36:25 +02:00
a8f9423307 Migrate chapter number to float 2024-01-22 13:45:36 +02:00
7c871edbca Merge pull request #503 from mangadventure/master
Add MangAdventure sources (Arc-Relight, Assorted Scans)
2024-01-22 13:14:35 +02:00
ObserverOfTime
f7d9579cee Relax pagination test 2024-01-20 19:27:16 +02:00
ObserverOfTime
b7e6ca8a26 Add MangAdventure sources
- Arc-Relight
- Assorted Scans
2024-01-20 19:27:16 +02:00
8852d1e22e [Remanga] Attempt to fix No value for page id 2024-01-19 23:31:21 +03:00
fcaa0ea442 Add volume field to chapter 2024-01-17 15:03:33 +02:00
8e7d7e0bde Add some sources 2024-01-15 01:58:21 +03:00
scaledzdn
50c6c9f842 src: id/shinigami.kt: Change url source (#481)
* Previous url got banned on Indonesia
2024-01-13 13:40:06 +03:00
a2979753a9 [Grouple] Fix pages parsing 2024-01-12 18:10:58 +02:00
789e39b6cb Fixes batch 2024-01-11 17:36:07 +02:00
2f5441ef20 [Remanga] Use home url as authUrl 2024-01-09 20:15:56 +03:00
devi
6ad08f13d4 Add Comic1000, ThunderScans, WhaleManga, SussyScan, EDoujin, 3XYaoi, SussyScan #444
Url TeamXNovel, WestManga
Remove AzureManga, BabelToon, FranxxMangas, LsHistoria, MangaGenki, Wowomik, MiraiScans, Nonbiri, PeaceScans, ReaperScansTr, TurkToon
2024-01-09 17:51:16 +01:00
devi
656f621711 Remove bad import 2024-01-07 14:51:15 +01:00
devi
6dc0e12fb0 add filter on sources
Remove Komikgo redirect to ManhwaLand
Add Template PizzaReaderParser and move FmTeam
Add PhoenixScans, TuttoAnimeManga, LupiTeam, LadyEstelarScan, GtoTheGreatSite
2024-01-07 14:50:16 +01:00
0e8579017b [BatoTo] Fix pages parsing 2024-01-07 12:02:13 +02:00
devi
e03d0efe71 Fix FlixScansOrg and add filter 2024-01-05 19:24:36 +01:00
0b497c4e0b [ComicK] Fix branches names 2024-01-05 16:22:32 +02:00
devi
704f589b6d Add new filter on sources
Url GekkouScans, SssScanlator
Add LunarScan #435
2024-01-04 19:10:22 +01:00
3feb84ac9e Update some domains 2024-01-04 13:12:14 +02:00
e375654009 [ComickFun] Fix branches and titles 2024-01-04 13:06:37 +02:00
ce7ac5d8f4 [ExHentai] Fix tags and branch 2024-01-04 13:02:56 +02:00
devi
a172fa5a5b Add new filter on sources
Add BurningScans
2024-01-03 19:33:45 +01:00
7c89f53988 [ExHentai] Improve tags loading 2024-01-03 11:54:56 +02:00
4e2d739840 [Shinigami] Add user-agent changing option 2024-01-02 20:34:37 +03:00
devi
5cdbda700b Urls : DomalFansub, HentaiVN, PheTruyen, TruyentranhLH
Add : KomikSay, MangaLesen
Remove : 1stManhwa, ConsejoDeMatones, OzulScans, SugarLab, HyoManga, Mi2Manga
2024-01-02 17:53:45 +01:00
b274b51699 Improve filtering 2024-01-02 14:17:20 +02:00
efb5d34279 Merge pull request #431 from KotatsuApp/ExcludeTags 2024-01-02 12:51:43 +02:00
devi
9ecb75b51c Add ZScanlation, KomikGesn , Kenhuav2Scan
Fix MangaMate
2023-12-31 11:41:45 +01:00
69332a85da Add isTagsExclusionSupported flag 2023-12-31 09:51:09 +02:00
ea095084cc Fix some parsers 2023-12-31 09:27:17 +02:00
1f176ebab2 Update dependencies 2023-12-31 08:32:20 +02:00
devi
327c05d03e Fix Chapters madtheme
Url NettruyenBing
Remove Dead Sources :
AkuManga , CmReader, FalconManga, JaiminisBox, Manga3s, MangaCv, MangaLeks, UniToonOficial , 1stKissManga.tv, Manhwa2read, ManhuaMix, ManhuaDex, Hwago, ImmortalUpdates, KomikSay, MangaTx, Scan-Fr.org, QueenScans, PhantomScans, OpScanlations, NeuManga.net, ManhwaFreak.fr, Lady Estelar Scan, KomikManhwa, KomikManga, KaratcamScans, Boosei, LyraScans, MangaDiyari, MangaPro, NHentai.uk, Oxapk, SpiderScans, MangaNelo.biz, MangaOnline.team, SpartanManga
2023-12-30 11:09:56 +01:00
devi
2d7c120d19 Add Feature Content Rating Filter 2023-12-29 14:59:50 +01:00
devi
a2e9f19814 Fix Build 2023-12-29 14:16:56 +01:00
devi
c602355e65 Feature Add Exclude Tags 2023-12-29 13:57:01 +01:00
devi
a390e0de49 Fix ALPHABETICAL Sort on BentomangaParser
Remove Toonily.net
Add MonzeeKomik, Tresdaos
2023-12-29 13:55:04 +01:00
a228d71d57 Merge pull request #429 from KotatsuApp/Alpha-zaAndStateUpcoming
Add a new filter and a new state
2023-12-28 16:43:35 +02:00
devi
b2f0eaccc4 Remove Import Local 2023-12-28 10:55:20 +01:00
devi
07aaec7d3c Add template Scan and move ScanVfOrg in
Add ScanIta

Remove Mangastic -> redirect to toon69
Remove MangaGoYaoi -> redirect to asurascans.us
Remove Cizgiromanarsivi -> redirect to mangaoku.info
Remove WebtoonEvreni -> redirect to merlinscans.com
Remove GalaxyAction -> redirect to flixscans.com
Remove GalaxyAction -> redirect to kingofshojo.com
Remove aquamanga.live -> redirect to kingofshojo.com
Remove MangaReaderPro -> redirect to hadesnofansub.com

Urls : S2Manga.io, KomikzoId, Toon69, ReaperComics, ShadowXManga, TeamXNovel, YaoiFlix, MangaEmpress, MangaMukai.com, komikRemaja.cfd, MangaKyo, KomikMirror, ManhwaDesu, ManhwaIndo.sbs, ManhwaLand, Wowomik, LireScanVf.com, NhattruyenPlus
2023-12-28 10:54:52 +01:00
devi
392e9af944 New SortOrder.ALPHABETICAL_DESC
New MangaState.UPCOMING
2023-12-26 15:17:40 +01:00
904e0719eb [HitomiLa] Fix tag names 2023-12-25 19:48:33 +02:00
52ac3d2e6c Merge pull request #428 from AwkwardPeak7/hitomi
HitomiLa: fix thumbnails
2023-12-25 19:21:54 +02:00
devi
859b07e454 Add Ler999, ShadowCeviri 2023-12-25 17:02:06 +01:00
AwkwardPeak7
d4b252fd71 HitomiLa: fix thumbnails
I dismissed this in their scripts as the thumbnails worked fine but now they broke
2023-12-25 19:28:41 +05:00
4a0e7221b0 [HitomiLa] Refactor 2023-12-25 13:08:19 +02:00
43bdbe5a01 Merge pull request #426 from AwkwardPeak7/hitomi
Hitomi.la
2023-12-25 12:51:17 +02:00
3d10456a87 Apply suggestions from code review 2023-12-25 12:50:56 +02:00
d5a6c95ffb [MangaInUa] Fix chapters loading 2023-12-25 12:40:31 +02:00
devi
edc07b5dbd New Template GuyaParser
Add DankeFursLesen, GuyaCubari, Hachirumi, MahouShoujobu, Olaoe
Add local on sources GattsuParser
2023-12-24 12:00:06 +01:00
devi
a5219ceb6c New Template GattsuParser
Add Baozimh, MuitoHentai, TopReadManhwa, HentaiSeason, HentaiTokyo, MundoHentaiOficial, UniversoHentai
2023-12-23 18:59:48 +01:00
AwkwardPeak7
039075086e hitomi: better locale in tag search 2023-12-23 22:57:12 +05:00
AwkwardPeak7
495c9fad33 hitomi: formatting 2023-12-23 22:23:43 +05:00
AwkwardPeak7
b61c5e8f12 hitomi: locales 2023-12-23 20:29:16 +05:00
AwkwardPeak7
a40a8d329a simplify search a bit 2023-12-23 19:55:21 +05:00
AwkwardPeak7
3f3ab5a7be Hitomi.la 2023-12-23 19:14:51 +05:00
devi
d8c32047d0 Fix MangaATrend
Add LerManga
Remove ReaperScansPt.kt
2023-12-22 17:23:52 +01:00
devi
f5f2854f36 Add OnePieceEx 2023-12-20 20:02:31 +01:00
devi
86a2b95141 Add Comicaso, LectorManga, Mangaoku, MangaSehri.net, MangaTr, TempestFansub.net
Fix KomikCast
Url ElarcPage , InfraFandub
2023-12-19 20:42:49 +01:00
4db7ccc853 Tiny refactoring 2023-12-19 08:11:19 +02:00
b1da5c2924 [Mangaowl] Fix chapters parsing 2023-12-19 07:45:56 +02:00
devi
0fe6fed838 Merge pull request #418 from PhantomShift/patch-1
Fix MangaDex nsfw check
2023-12-17 18:09:23 +01:00
PhantomShift
a3f8c0a236 Fix MangaDex nsfw check
Additionally check for pornographic content rating when labeling a work as NSFW.
2023-12-17 10:49:12 -05:00
devi
c88d7de138 Add AComics, ReaperComics ( en ) 2023-12-17 14:48:54 +01:00
devi
eeafb3c6fd Fix JiangzaiToon #417 2023-12-17 11:58:29 +01:00
devi
0208f5be97 Add DatgarScanlation, HyoManga, HeckScans, MerlinScans, TraduccionesMoonlight, NinjaScan, Mi2MangaEs, ScanVfOrg
Url InariManga
Fix chapters MangaGeko
2023-12-16 17:24:54 +01:00
42ecf8a958 Merge pull request #413 from KotatsuApp/ZeistMangaParser
Add ZeistMangaParser
2023-12-16 08:06:03 +02:00
d780623b98 Merge pull request #406 from AwkwardPeak7/mplus
MangaPlus
2023-12-16 07:49:17 +02:00
devi
1b1a167580 Add EpikMan 2023-12-15 23:06:29 +01:00
devi
49d0c80070 Add MaxgsScan, WolfScanBr 2023-12-15 22:51:27 +01:00
devi
a35c2f3394 Add ZeistMangaParser and sources
add urldecode
ul VfScan
2023-12-15 22:04:47 +01:00
AwkwardPeak7
3034a87929 MangaPlus: divide different languages to avoid too many calls during chapter fetching & prevent some non-english titles to be filtered out 2023-12-15 18:34:27 +05:00
AwkwardPeak7
33b96043c9 Merge remote-tracking branch 'upstream/master' into mplus 2023-12-15 18:07:24 +05:00
devi
4d02dc0d12 Move FlixScansOrg to En
add MangasOnline , WorldManhwas
Url Shinigami
2023-12-14 18:18:18 +01:00
AwkwardPeak7
79fd2a58c1 MangaPlus: improvements 2023-12-14 19:19:32 +05:00
AwkwardPeak7
ab82d0eba0 MangaPlus 2023-12-13 21:18:57 +05:00
75cc0716fd Merge pull request #404 from AwkwardPeak7/999 2023-12-13 17:59:59 +02:00
87f99addbb Update SuspendLazy 2023-12-13 12:59:26 +02:00
AwkwardPeak7
3e1b15a23c review changes 2023-12-12 23:35:07 +05:00
devi
5758e9f68f Add BentoScan, Manga18.xyz, NewManhua
Url MangaScan, Anime-Sama.me
Fix DoujinDesu.tv
2023-12-12 18:29:01 +01:00
AwkwardPeak7
0964c02a44 largeCover and better chapter branch name 2023-12-12 18:55:15 +05:00
AwkwardPeak7
bf06ab7492 use SuspendLazy for fetching cdn host 2023-12-12 17:27:38 +05:00
AwkwardPeak7
e992566899 999Hentai 2023-12-12 14:01:36 +05:00
devi
97338f31c5 Add IrisScanlator #403
Add RagnaScan
Fix VexManga
2023-12-11 17:21:44 +01:00
devi
0ff4f976e1 Fix FlixScans
Add FlixScansOrg
Add DoujinKu
2023-12-10 20:28:35 +01:00
devi
496b3388ee Add source : MangaPark , GekkouScans, YuriLab 2023-12-09 17:19:43 +01:00
devi
d6b578acb0 Add, fix sources
Manual addition of lang on Bato.To
Add multi tag , lang and rework tags on exhentai
Add lang on imhentai
2023-12-08 17:18:23 +01:00
b3be87f541 Update readme 2023-12-08 07:27:35 +02:00
devi
786cd12dba Add, fix sources
Add language on Bato.To
2023-12-06 19:24:15 +01:00
devi
a1a17c02d7 Add, fix sources
Add language on Bato.To
2023-12-06 19:23:46 +01:00
devi
b8caf8e572 Minor update. 2023-12-05 17:37:07 +01:00
44c2c074a8 [MangaDex] Language filter support 2023-12-05 13:39:07 +02:00
3ed960254d [GalleryAdults] Language filter support #357 2023-12-05 13:05:11 +02:00
f1e8fbec5d Merge pull request #401 from KotatsuApp/filter
Rework getList
2023-12-05 11:26:17 +02:00
c5ed9cf54b Merge pull request #400 from deividgabrielpeira/master
Neoxscan change url
2023-12-05 11:25:03 +02:00
devi
65c567dbcb Fix NHentaiParser 2023-12-04 18:16:56 +01:00
devi
b312823afd Merge remote-tracking branch 'origin/master' 2023-12-03 12:57:14 +01:00
devi
56eb059bcd fix getlist VmpParser, LineWebtoonsParser, BlogTruyenParser, HentaiVNParser, YurinekoParser
add filter on WpComicsParser, ZMangaParser, LxManga, Truyenqq, TruyentranhLHParser

Move NetTruyen to template WpComicsParser and add multitags on WpComicsParser
2023-12-03 12:56:52 +01:00
devi
6ad78f1cba fix getlist OtakuSanctuaryParser, bakai , brmangas, LerMangaOnline, MangaOnline, YugenMangas, MangaAy, SadScans, YaoiFlix
add filter on SinmhParser, TrWebtoon
2023-12-02 16:41:37 +01:00
devi
26b758ae50 fix getlist some source mangaeader, MmrcmsParser
add filter on NepnepParser
2023-12-01 18:50:34 +01:00
devi
019b8cc087 fix getlist some source mangaeader, MmrcmsParser
add filter on NepnepParser
2023-12-01 18:49:30 +01:00
Deivid Gabriel Pereira de Oliveira
c95845083a Neoxscan change url 2023-12-01 16:48:41 +00:00
9c7f314f65 [HentaiUkr] Add usage of objects69.json 2023-12-01 02:48:00 +03:00
devi
743a92a3e8 remove import 2023-11-30 18:46:52 +01:00
devi
37e9b33c24 add filter : LikeMangaParser, FmTeam, MadthemeParser , MangaReaderParser
Fix GetList : NicovideoSeigaParser, Manga18Parser

add filter & multitags on MangaboxParser
2023-11-30 18:46:10 +01:00
devi
7c2ac033d7 add filter on HeanCms , DoujinDesu.tv
Fix GetList GalleryAdultsParser
2023-11-29 18:31:26 +01:00
0efd5437f9 [HentaiUkr] Fix objects loading 2023-11-29 17:18:01 +02:00
devi
db770351f3 add filter on FmreaderParser, LugnicaScans
Fix GetList on FoolSlideParser , FuryoSociety , LegacyScans, LireScan, ScansMangas.me, ScantradUnion
2023-11-28 21:07:54 +01:00
devi
14b2457627 fix getlist TempleScanEsp
fix getlist and add tag on Details mangas TuMangaOnline
2023-11-27 18:57:47 +01:00
devi
f62ea43de8 fix getlist on AnimeBootstrapParser, ImHentai, beetoon, CloneManga, MangaGeko, Po2Scans, Pururin
add filter FlixScans, TeamXNovel, ComicExtra, MangaOwl.to, MangaTown, Manhwa18.net, ManhwasMen

fix getlist and add tag list on DynastyScans
fix getlist and add SortOrder.UPDATED on MangaStorm
2023-11-26 18:44:34 +01:00
devi
ab2aad6bd0 add filter on bentomanga
add filter on ninemanga
2023-11-25 20:52:02 +01:00
devi
1293233ad9 fix getListPage Some sources madara 2023-11-25 20:04:47 +01:00
devi
5fff2adbfb MadaraParser add filter.states
- Some sources madara, need to be changed too
2023-11-25 17:57:38 +01:00
devi
7d67b718a5 Mangadex add filter.states and info states 2023-11-25 17:45:31 +01:00
devi
b8ca21d588 unwanted change 2023-11-25 17:37:04 +01:00
devi
44d91e43ad Add altName on ComicK
Change oneOrThrowIfMany on Bato.To
2023-11-25 17:34:52 +01:00
3bb10ee948 [HentaiUkr] Url hotfix 2023-11-25 19:22:42 +03:00
devi
81caa59a36 Add status filter support
Add oneOrThrowIfMany() on MangaState
2023-11-25 17:06:21 +01:00
46e863ef79 [ComicK] Fix manga state 2023-11-25 17:12:44 +02:00
cecba7623e [ComicK] Fixes and status filter support 2023-11-25 14:44:04 +02:00
7dee865c8a Merge pull request #397 from deividgabrielpeira/master
Daily Update (extensions portuguese)
2023-11-25 13:06:57 +02:00
Deivid Gabriel Pereira de Oliveira
5ac3cdb03c Arkham Scan -> HuntersScan (Fusion) 2023-11-24 12:28:54 +00:00
c3613f3ba4 [Grouple] Migrate to the new getList 2023-11-23 12:20:30 +02:00
e7ac1c1feb Fix PagedMangaParsers 2023-11-23 12:13:32 +02:00
24573d12f5 Fix compiling 2023-11-23 11:32:14 +02:00
9d2f633e14 Merge branch 'feature/filter-ex' 2023-11-23 11:20:26 +02:00
570b0c19c5 Merge pull request #388 from deividgabrielpeira/master
Daily Update (extensions portuguese)
2023-11-23 11:15:34 +02:00
Deivid Gabriel Pereira de Oliveira
a11e42a4d6 Merge branch 'KotatsuApp:master' into master 2023-11-19 14:50:31 -03:00
devi
410c73a74f Add multi tag some sources
Add SortOrder.POPULARITY some sources
Add all language on GalleryAdultsParser
fix YugenMangas (pt)
2023-11-19 17:58:10 +01:00
devi
3ea088371b Minor daily update 2023-11-18 20:44:35 +01:00
Deivid Gabriel Pereira de Oliveira
09a0bb364a HentaiSSSSSS (is closed) 2023-11-18 17:30:45 +00:00
Deivid Gabriel Pereira de Oliveira
d91947ae15 NocturneSummer fix? 2023-11-18 17:21:44 +00:00
Deivid Gabriel Pereira de Oliveira
0a8a9e5935 NocturneSummer fix 2023-11-18 17:18:31 +00:00
Deivid Gabriel Pereira de Oliveira
59226b298f apenasmaisumyaoi fix 2023-11-18 17:16:24 +00:00
Deivid Gabriel Pereira de Oliveira
dc41bf3e83 ArthurScan Fix 2023-11-18 16:06:30 +00:00
Deivid Gabriel Pereira de Oliveira
a790a18f7f test 2023-11-18 16:04:21 +00:00
Deivid Gabriel Pereira de Oliveira
1fcc1096f5 ArthurScan add Catalog (pt) and (test) 2023-11-18 15:59:47 +00:00
Deivid Gabriel Pereira de Oliveira
86aaca271f Merge branch 'KotatsuApp:master' into master 2023-11-18 12:41:55 -03:00
Deivid Gabriel Pereira de Oliveira
b8cc2d6953 Animaregia (Down) 2023-11-18 14:48:56 +00:00
11f110b0bb Merge pull request #385 from KotatsuApp/add-hiatus
add hiatus
2023-11-18 16:09:02 +02:00
Deivid Gabriel Pereira de Oliveira
31459ea6e7 yaoitoshokan (closed the site) 2023-11-18 12:58:48 +00:00
Deivid Gabriel Pereira de Oliveira
7ab84688c2 LerYaoi (down), posting in telegram 2023-11-18 12:56:12 +00:00
Deivid Gabriel Pereira de Oliveira
d18f287804 NocturneSummer fix 2023-11-18 12:44:41 +00:00
Deivid Gabriel Pereira de Oliveira
1c3b41acba Nocsummer, yaoi site, necessary (ContentType.HENTAI) 2023-11-18 12:42:15 +00:00
Deivid Gabriel Pereira de Oliveira
ecfc7e40c9 HiperCool, change url 2023-11-18 12:29:55 +00:00
41eea1c420 Set jvmTarget to 8 2023-11-18 14:28:13 +02:00
Deivid Gabriel Pereira de Oliveira
594524afc3 apenasmaisumyaoi change name, possibly the url will be changed 2023-11-18 12:16:12 +00:00
Deivid Gabriel Pereira de Oliveira
70a5337604 AkumanoTenshi -> tatakaeScant (fusion) 2023-11-18 12:12:02 +00:00
Deivid Gabriel Pereira de Oliveira
924fde8790 CeriseScans, change url 2023-11-18 11:58:45 +00:00
dea70974bf Revert unwanted changes 2023-11-18 13:03:09 +02:00
43fd7b8d47 [HoneyManga] Fix images urls 2023-11-18 12:55:02 +02:00
devi
6c2a526aeb rename 2023-11-18 11:27:38 +01:00
43a8caa687 Advanced filtering support 2023-11-18 10:22:35 +02:00
devi
f3fc848b6c Minor daily update 2023-11-17 17:54:57 +01:00
devi
98f5d115b3 Minor daily update 2023-11-16 17:55:13 +01:00
devi
885d98cebd Minor daily update 2023-11-15 18:37:47 +01:00
devi
6f73ff23e8 Minor update 2023-11-14 20:48:06 +01:00
2cd2d7dbc4 Merge pull request #368 from deividgabrielpeira/master
Update Daily
2023-11-14 07:31:22 +02:00
devi
d3afdad95b Minor update 2023-11-12 12:35:41 +01:00
Deivid Gabriel Pereira de Oliveira
48ac46eb5c chance url VexManga : Source url changed #370 2023-11-11 14:52:13 +00:00
Deivid Gabriel Pereira de Oliveira
2d6e6b359b they'll shut down leitorizakaya, they'll switch to blackoutcomics, it makes more sense to put it on madara, because they've made their own website. 2023-11-11 14:00:31 +00:00
Deivid Gabriel Pereira de Oliveira
7462743be8 se não funciona, so na proxima 2023-11-10 15:01:07 +00:00
Deivid Gabriel Pereira de Oliveira
db2acb752a fix 2023-11-10 14:55:37 +00:00
Deivid Gabriel Pereira de Oliveira
335229e61b fix error thanks @ztimms73 2023-11-10 14:48:25 +00:00
Deivid Gabriel Pereira de Oliveira
2acdd66f70 fix again 2023-11-10 14:42:42 +00:00
Deivid Gabriel Pereira de Oliveira
d3bec9006e Fix uppercase in parser 2023-11-10 14:40:23 +00:00
Deivid Gabriel Pereira de Oliveira
44f70c1388 Merge branch 'KotatsuApp:master' into master 2023-11-10 11:08:33 -03:00
02463e5833 [MangaReader] Fix NetShield bypass 2023-11-10 13:06:13 +02:00
Deivid Gabriel Pereira de Oliveira
778d14a1a2 change mangaSource 2023-11-09 22:27:19 +00:00
Deivid Gabriel Pereira de Oliveira
8b3814c485 , 2023-11-09 20:21:28 +00:00
Deivid Gabriel Pereira de Oliveira
6bcd9dd65c mudou de nome 2023-11-09 20:18:26 +00:00
154ae09c6e [MangaReader] Add NetShield bypass (testing) #198 2023-11-07 18:33:22 +02:00
devi
58c9d14d42 Minor daily update 2023-11-06 20:24:12 +01:00
3e69b88518 Update dependencies 2023-11-06 10:42:04 +02:00
devi
9d3671e117 Minor daily update 2023-11-05 18:07:37 +01:00
devi
0731d2487a Minor daily update 2023-11-05 18:07:12 +01:00
devi
5e4dd82b8d Minor daily update 2023-11-04 18:56:46 +01:00
devi
9d33d1a560 Minor daily update 2023-11-03 18:28:00 +01:00
devi
e788cff04a Minor daily update 2023-11-02 19:36:26 +01:00
face1d5b26 Merge pull request #355 from KotatsuApp/GalleryAdultsParser
Add GalleryAdultsParser
2023-11-02 08:52:39 +02:00
devi
18f8472baa Add GalleryAdultsParser
reworking some sources
add sources
2023-11-01 21:28:36 +01:00
devi
f14edd871f Minor daily update 2023-11-01 17:22:52 +01:00
c892a38c52 Trigger rebuild 2023-11-01 16:25:49 +02:00
devi
e9d473711d Minor daily update
- Fix tag ninemanga
- Fix search neonep parsser
2023-10-31 19:03:28 +01:00
devi
97a8bb1ca7 Minor daily update 2023-10-30 18:57:42 +01:00
devi
66d47bf579 Minor daily update 2023-10-29 18:31:35 +01:00
devi
508647eb5a Minor daily update ( and remove warning for name ) 2023-10-28 22:22:01 +02:00
4ca3a492b0 [ComicK] Fix genres 2023-10-28 12:40:00 +03:00
devi
fffe104cbe Minor daily update 2023-10-27 18:48:33 +02:00
devi
6258476a58 Minor daily update 2023-10-26 18:11:42 +02:00
devi
af1aca8725 Merge remote-tracking branch 'origin/master' 2023-10-25 17:51:04 +02:00
devi
bdb7fbc039 Minor daily update 2023-10-25 17:50:58 +02:00
6bf0ae92e4 [*chan] Fix sort orders 2023-10-25 13:36:42 +03:00
17b8fd51b7 [ComicK] Fix chapters numbers 2023-10-25 12:53:48 +03:00
d089fad6ce [MangaLib] Add branching support 2023-10-25 12:48:44 +03:00
ec11b79b18 [MangaLib] Fix search result covers 2023-10-25 12:22:10 +03:00
0f216c6de9 [Remanga] Fix loading large amount of chapters 2023-10-25 12:09:41 +03:00
devi
0606ace83a Merge pull request #328 from deividgabrielpeira/master
HentaiGekkout(pt)
2023-10-24 17:04:56 +02:00
Deivid Gabriel Pereira de Oliveira
746d857550 I forgot about Hentai Gekkou, which is part of Gekkou, which was taken down together by the DMCA. 2023-10-24 12:04:43 +00:00
devi
cf1d0b8d7e Merge pull request #327 from deividgabrielpeira/master
GekkouScan(pt) and FinalScans(pt), is down
2023-10-24 08:20:42 +02:00
Deivid Gabriel Pereira de Oliveira
d94145c6e5 GekkouScan(pt), has been taken down by the DMCA
FinalScans(pt), just closed
2023-10-24 00:42:56 +00:00
devi
b71a0ac3cf Minor daily update 2023-10-23 18:24:51 +02:00
devi
32afa26be5 Minor daily update 2023-10-22 20:15:17 +02:00
devi
23cc86e429 Minor daily update 2023-10-21 21:20:48 +02:00
devi
87032a6835 Minor daily update 2023-10-20 15:28:14 +02:00
devi
a4d9c6e36c Minor daily update 2023-10-19 18:46:07 +02:00
devi
76af0b0edb Minor daily update 2023-10-17 18:49:33 +02:00
0054d06e6e Fix sources titles 2023-10-17 09:49:38 +03:00
devi
e2e027df81 Minor daily update 2023-10-16 19:54:49 +02:00
a61e441e79 Merge pull request #283 from KotatsuApp/devi4
add sources and fix
2023-10-16 12:39:36 +03:00
devi
9b4153aacc fix 2023-10-15 10:12:20 +02:00
devi
b1d2d0a475 add Fakku 2023-10-14 16:43:42 +02:00
devi
aeec98db2a Rework tag hentaivn 2023-10-14 10:58:47 +02:00
devi
00f21de9eb quick fix 2023-10-14 10:28:30 +02:00
devi
2d595bb6ee Add small sources, fix , rename sources , remove code not used, remove dead sources, typo. 2023-10-14 10:10:21 +02:00
devi
e43c9f6489 Add small sources, fix , rename sources , remove code not used, remove dead sources, typo. 2023-10-14 10:08:00 +02:00
devi
81004b53a9 add Yaoi Flix 2023-10-13 18:38:36 +02:00
devi
85a0918da5 add Manga Ay 2023-10-13 17:38:02 +02:00
devi
b10fa0fd10 add change requested and fix url 2023-10-13 16:15:19 +02:00
a2f4bb35cd [*chan] Fix title parsing 2023-10-13 13:34:07 +03:00
devi
4120514256 add search on manga online 2023-10-12 19:39:15 +02:00
devi
29ef717ece add sources and fix 2023-10-12 18:56:49 +02:00
devi
d34cfbf405 add sources and fix 2023-10-12 18:56:20 +02:00
devi
94c2326153 Rework Yugen Mangas 2023-10-11 19:49:59 +02:00
devi
06c2287b71 fix 2023-10-11 17:47:14 +02:00
devi
8cb0d3d6fe Tag reduction on Pururin 2023-10-10 17:24:22 +02:00
devi
76860f82b9 add sources and fix 2023-10-10 17:17:06 +02:00
devi
f134137a05 add sources and fix 2023-10-09 17:58:43 +02:00
devi
fc231ec330 fix Asura Scans 2023-10-08 21:28:13 +02:00
devi
d3c32faf01 Tag reduction on hentaifox 2023-10-08 18:38:32 +02:00
devi
d16f4cd892 add ImHentai 2023-10-08 17:31:01 +02:00
devi
9a8de28a1d add Pururin 2023-10-08 15:29:25 +02:00
devi
939f08b8a1 add Mirai Scans 2023-10-08 09:55:59 +02:00
devi
09500c0734 add sources and fix 2023-10-07 17:10:29 +02:00
400a90464e [LineWebtoons] Refactoring 2023-10-06 10:36:51 +03:00
9103f9d473 Merge pull request #273 from DCNick3/line-webtoons-source
Add Line Webtoons sources (webtoons.com)
2023-10-06 09:47:45 +03:00
devi
27c29f99d9 minor fixes and add small sources 2023-10-04 17:14:12 +02:00
Nikita Strygin
753c27a90c Clean up code after review
- Do not use ConfigKey.Domain for supplementary domains
- use IllegalArgumentException
- use "$var" instead of "${var}" where possible
- fix the language in `publicUrl`
- use `?` to handle `largeCoverUrl`
- use `requireNotNull` instead of `!!`
2023-10-04 15:19:27 +03:00
3d7e62d2fe [MintManga] Update domain 2023-10-04 14:02:09 +03:00
e8289277a7 Merge pull request #274 from KotatsuApp/devi3
Add sources
2023-10-04 13:21:54 +03:00
Nikita Strygin
887fc691d1 Add support for more languages in Line Webtoons parser
Also change the default sorting to POPULARITY & clean up
2023-10-04 12:03:21 +03:00
devi
543e3da194 Add sources 2023-10-03 18:25:43 +02:00
devi
908cc22619 Minor fix 2023-10-03 16:55:32 +02:00
Nikita Strygin
c09d484a14 Add Line Webtoons sources (webtoons.com)
It uses the API of mobile app
2023-10-03 09:15:39 +03:00
07c3f22f75 Merge pull request #268 from VietAnh14/source/yurineko 2023-10-02 20:08:08 +03:00
devi
d494bf19ee Small fix 2023-10-02 17:29:07 +02:00
vianh
eda4f75219 [Yurineko] Fix api domain and chapters order 2023-10-02 21:21:58 +07:00
931f126119 Merge pull request #269 from deividgabrielpeira/patch-1
prismascan chance Domain
2023-10-02 13:19:59 +03:00
bc171ce06e [Grouple] Fix pumpkins 2023-10-02 13:19:36 +03:00
Deivid Gabriel Pereira de Oliveira
191d1fcec8 prismascan chance Domain 2023-10-01 21:01:05 -03:00
vianh
eaea9e54d1 [Yurineko] New source 2023-10-01 20:27:51 +07:00
devi
c9b3912fd4 Quick small fix 2023-10-01 00:01:20 +02:00
1611dc3429 Remove duplicated classes 2023-09-30 17:24:47 +03:00
88dc2145e0 Merge branch 'AwkwardPeak7-ksk' 2023-09-30 17:09:27 +03:00
8979987344 Merge pull request #266 from KotatsuApp/devi2
Update focus on fix and remove dead sources
2023-09-30 17:05:04 +03:00
AwkwardPeak7
1cec2cc187 remove need for refreshing chapter list 2023-09-29 20:29:14 +05:00
devi
a1279b64a2 small fix and sources 2023-09-29 17:21:38 +02:00
AwkwardPeak7
2ce5f2563a KskMoe: get original quality images 2023-09-29 19:29:57 +05:00
devi
75a60ff738 fix Union Mangás 2023-09-28 20:04:46 +02:00
devi
74eb7f83d6 other fix 2023-09-28 17:24:02 +02:00
devi
5b2ac5f6fb typo 2023-09-27 18:51:10 +02:00
devi
fde4ba1d85 remove mangatone 2023-09-27 18:45:56 +02:00
devi
8278a8f5c3 multiples fix, remove dead source and add source 2023-09-27 18:44:09 +02:00
a39d644094 [Remanga] Ability to change user-agent 2023-09-27 17:11:00 +03:00
0004be15ba Merge branch 'source/devi' 2023-09-20 12:11:03 +03:00
8fef459346 Misc fixes 2023-09-20 12:09:54 +03:00
109603f124 [HentaiUkr] Fix ids 2023-09-20 10:04:15 +03:00
91879a8405 Merge pull request #261 from CakesTwix/master
[HentaiUkr] Tag ID is String now
2023-09-20 09:43:33 +03:00
devi
f471a3513e add mangashiro.me and fix 2023-09-19 18:48:26 +02:00
CakesTwix
7362111318 [HentaiUkr] Tag ID is String now 2023-09-19 11:29:52 +03:00
devi
609c7188c0 add sources tr 2023-09-18 19:48:24 +02:00
devi
a106b34379 add sources and fix 2023-09-18 18:39:05 +02:00
devi
853b95363e add missing content type 2023-09-17 19:49:21 +02:00
devi
45113e419e small fix 2023-09-17 17:18:09 +02:00
devi
519bb3053b add ksk 2023-09-17 16:57:50 +02:00
devi
366cc862ba remove 1stkissmanga.me and add likemanga 2023-09-17 13:12:35 +02:00
devi
611aaf0d38 remove mirrordesu 2023-09-17 10:52:27 +02:00
devi
712d829b54 add sources 2023-09-17 10:50:01 +02:00
devi
e25656c090 reformat 2023-09-17 09:35:58 +02:00
devi
7debe6d7e2 add sources 2023-09-16 17:07:59 +02:00
devi
55e14e4cb3 add sources 2023-09-16 17:07:22 +02:00
7fbeb2e266 Add gradle jvmargs 2023-09-12 19:22:36 +03:00
7be6d779da Reformat 2023-09-12 19:15:57 +03:00
96585fbdf7 [DesuMe] Add alternate domain 2023-09-12 18:27:04 +03:00
1a0d623c02 Merge pull request #252 from davvarrr/master
adds sources and fix
2023-09-12 18:25:55 +03:00
67cf2c2a6e Revert MangaSources changes 2023-09-12 18:24:13 +03:00
b6dd7fb5f6 [HentaiUkr] Fixes 2023-09-12 18:23:34 +03:00
e0c5562600 Merge pull request #251 from CakesTwix/master
New 18+ UA Source and update urls in `CONTRIBUTING.md`
2023-09-12 13:47:20 +03:00
devi
2447f66834 change domain 2023-09-10 09:33:10 +02:00
devi
11ed27e397 correction 2023-09-09 18:12:19 +02:00
devi
6158547571 correction 2023-09-09 18:10:16 +02:00
devi
a24dfde1ba correction 2023-09-09 18:09:58 +02:00
devi
d805b49a4d adds sources and fix 2023-09-09 17:12:20 +02:00
devi
f871bf77fa Merge remote-tracking branch 'origin/master' 2023-09-09 17:11:36 +02:00
devi
be610b825d adds sources and fix 2023-09-09 17:11:14 +02:00
f791cc6f9c Added SoftSuspendLazy class 2023-09-09 16:36:13 +03:00
CakesTwix
30b2e81b15 [CONTRIBUTING] Update example urls
380861f29e
2023-09-08 20:51:36 +03:00
CakesTwix
067c7f2d0d [HentaiUkr] New 18+ UA Source 2023-09-08 20:51:25 +03:00
aae3fa3b05 Add abandoned state 2023-09-08 13:19:26 +03:00
41cd8ac7d7 Merge pull request #250 from CakesTwix/master
[HoneyManga] Change search url
2023-09-08 13:09:34 +03:00
CakesTwix
61f2a499da [HoneyManga] Change search url 2023-09-08 12:46:18 +03:00
2f7e704e21 [MangaLib] Check page response size 2023-09-03 18:55:55 +03:00
4d22e7d1e8 Merge pull request #243 from davvarrr/master
Add sources and fix
2023-09-03 18:30:44 +03:00
devi
18125ea982 fix url Ozulscans 2023-09-03 09:52:55 +02:00
devi
1498541b45 fix url Nivera Fansub 2023-09-02 16:27:24 +02:00
devi
4faaa9fc13 fix url Manhwalist 2023-09-02 13:35:27 +02:00
devi
c6c4b7190a fix url Random Scans 2023-09-02 10:09:33 +02:00
devi
c0bf3ad56c add RackusReads 2023-09-01 12:28:10 +02:00
devi
ad51271c40 format code 2023-09-01 12:14:32 +02:00
devi
fce952d240 change request and change url Nettruyenmax & NetTruyen 2023-09-01 12:13:57 +02:00
devi
0dc5032a7a change url Shinigami 2023-08-31 19:22:51 +02:00
devi
0395f29503 small change 2023-08-31 12:31:28 +02:00
devi
03581add21 add Perf Scan and fix Legacy Scans 2023-08-31 12:26:13 +02:00
devi
e0249accf8 fix domain Mgkomik 2023-08-31 09:46:19 +02:00
devi
a3afbafdcc remove small code 2023-08-30 18:03:45 +02:00
devi
4d4c126327 remove dead source 2023-08-30 17:50:26 +02:00
devi
c4bf7b2ae0 add sources and fix 2023-08-30 13:45:40 +02:00
devi
5d5f804c76 add sources and fix 2023-08-30 13:44:25 +02:00
3a76504380 [NepNep] Fixes 2023-08-24 14:16:47 +03:00
5035c90c73 Merge pull request #238 from davvarrr/master
add source and move Manga4Life to tempalte parser
2023-08-24 13:55:16 +03:00
devi
7f1cd30ce8 change domaine for QueenScans 2023-08-21 20:49:36 +02:00
devi
155a91a13d add Mangarbic 2023-08-21 12:19:54 +02:00
devi
8c291e274d nothing 2023-08-21 10:36:38 +02:00
devi
10e1eb8aac add source and move Manga4Life to tempalte parser
and fix url Duniakomik
2023-08-21 10:33:47 +02:00
06a2aa6f97 [Manga4Life] Refactoring and fixes 2023-08-19 17:38:57 +03:00
f08a2e4551 Merge pull request #235 from davvarrr/master
add sources and fix
2023-08-19 17:18:05 +03:00
e6839b1270 [Grouple] Change getPageUrl exception 2023-08-19 16:53:42 +03:00
270890f841 [NudeMoon] Migrate to mobile website parsing 2023-08-19 16:51:03 +03:00
devi
02581c9e2d Change and add Manga4Life 2023-08-19 15:38:40 +02:00
devi
ea4e69df19 remove japscan 2023-08-19 08:40:03 +02:00
devi
cb0ebb31f7 add more source 2023-08-18 20:03:31 +02:00
devi
497cd0bd15 add more source 2023-08-18 19:58:01 +02:00
devi
b74c7841dd add sources and fix 2023-08-18 17:39:35 +02:00
407ef5b655 Merge pull request #232 from davvarrr/master
quick fix
2023-08-14 09:54:20 +03:00
devi
f034b1874b add Mangakakalot 2023-08-13 11:26:29 +02:00
devi
eac086b935 add jaiminisbox 2023-08-13 09:26:20 +02:00
devi
ea310b184e add MangaCrab and MantrazScan 2023-08-13 09:20:15 +02:00
devi
ea3386eda5 add MangaCrab and MantrazScan 2023-08-13 09:19:52 +02:00
devi
2efdb1940a fix Asura Scans 2023-08-13 09:00:57 +02:00
devi
896139a8ed Add exception for source don't support multi tags
and other fix
Add birdToon
2023-08-12 15:24:27 +02:00
devi
da344233f4 quick fix 2023-08-11 18:30:48 +02:00
9cf3551356 [HentaiVN] Fix async/await 2023-08-11 12:01:32 +03:00
10aaae1ca8 Merge pull request #229 from davvarrr/master
Add template FoolSlideParser ands sources
2023-08-11 11:44:19 +03:00
dcb5206272 Add minus operator to favicons 2023-08-11 11:20:27 +03:00
devi
ccdd2b6c19 fix TuMangaOnlineParser 2023-08-10 17:47:01 +02:00
devi
16f3e1c180 add Ninja Scan 2023-08-09 21:10:24 +02:00
devi
7283cde5ff add missing contenttype 2023-08-09 21:01:43 +02:00
devi
4c20384d39 add sources and small fix 2023-08-09 20:59:03 +02:00
devi
5e9bc4fcc7 fix async and remove .imageUrl() to add .src() 2023-08-09 19:49:19 +02:00
devi
e5448649c3 fix pagination HentaiVNParser 2023-08-07 22:27:18 +02:00
devi
951e89d429 add rating on MadaraParser 2023-08-07 19:24:56 +02:00
devi
f34dd7926a add skanlacje-feniksy and fix hentaivn 2023-08-07 17:44:25 +02:00
devi
b900c05d9b Merge remote-tracking branch 'origin/master' 2023-08-06 22:45:34 +02:00
devi
06948934ac and sources 2023-08-06 22:45:14 +02:00
devi
e82915ef17 Merge branch 'KotatsuApp:master' into master 2023-08-06 17:56:52 +02:00
devi
bb37da403f Add template FoolSlideParser ands sources 2023-08-06 17:56:17 +02:00
065263a9bd Merge pull request #227 from davvarrr/master 2023-08-06 10:35:45 +03:00
devi
f3359cc830 quick fix hentai20 move to tempalte mangareader 2023-08-06 09:22:00 +02:00
1b3b5d95c0 Merge pull request #226 from davvarrr/master
fix and add sources
2023-08-06 09:00:10 +03:00
c290ba5436 Update src/main/kotlin/org/koitharu/kotatsu/parsers/site/en/DynastyScans.kt 2023-08-06 09:00:04 +03:00
devi
bb4ea05375 fix domaine peacescans 2023-08-05 18:18:25 +02:00
devi
47dde65f65 fix and add sources 2023-08-05 18:04:13 +02:00
08eb9552d6 Update gradle, kotlin and enum extensions 2023-08-04 15:58:57 +03:00
6058221fec Merge pull request #223 from davvarrr/master
add templates and sources and fix
2023-08-02 10:05:16 +03:00
devi
5bf4ecb394 min change 2023-07-31 20:28:01 +02:00
devi
41fa80e8b8 Change .src() in accord of commit 3c43590 2023-07-31 20:22:56 +02:00
devi
0dd3a26080 no pagination 2023-07-31 20:12:37 +02:00
devi
fb400d83e3 q fix 2023-07-31 19:54:21 +02:00
devi
37e5c43e5c Merge branch 'KotatsuApp:master' into master 2023-07-31 19:50:21 +02:00
341aa0e13d [JapScan] Migrate to desktop version 2023-07-31 15:08:46 +03:00
3c4359096d Add Jsoup extensions 2023-07-31 13:01:59 +03:00
devi
d56240086c Update src/main/kotlin/org/koitharu/kotatsu/parsers/site/zmanga/ZMangaParser.kt
Co-authored-by: Koitharu <nvasya95@gmail.com>
2023-07-31 11:58:33 +02:00
devi
bfea5b314e Update src/main/kotlin/org/koitharu/kotatsu/parsers/site/fr/ScansMangasMe.kt
Co-authored-by: Koitharu <nvasya95@gmail.com>
2023-07-31 11:58:22 +02:00
03b4fc9f00 Add Result utils 2023-07-31 09:53:21 +03:00
devi
826563694e add MangaboxParser and sourcres 2023-07-30 21:53:10 +02:00
devi
880ff923bc add AnimeBootstrapParser and sources 2023-07-30 15:11:44 +02:00
devi
cd0d4b103a add zmanga and soruces 2023-07-30 12:31:28 +02:00
devi
fefec4985d add Wpcomics and sources 2023-07-29 16:18:58 +02:00
devi
ba655f3d64 add domain and fix search url 2023-07-29 09:30:44 +02:00
devi
be3812fa2f fix pagination komikcast 2023-07-29 09:04:25 +02:00
devi
87d4326411 add MadthemeParser and sources 2023-07-29 08:11:51 +02:00
devi
ef6ede6b12 add correct page for UPDATED 2023-07-28 17:07:29 +02:00
devi
eeccda24b0 Add 2 source more typo and add val on gettags on madara for site not let list /manga/ 2023-07-28 16:53:17 +02:00
devi
2bf29da2ea small fix and rename bad typo 2023-07-28 16:16:13 +02:00
devi
0b54029e26 Merge remote-tracking branch 'origin/master' 2023-07-28 15:58:14 +02:00
d24ab347cb Move CloudFlareProtectedException to test sources 2023-07-28 14:35:13 +03:00
74ffe9418b Fix html descriptions 2023-07-28 10:49:20 +03:00
c945434d71 Merge pull request #222 from davvarrr/master
add new template and source
2023-07-28 10:38:22 +03:00
devi
b28545c852 add sources 2023-07-27 20:50:53 +02:00
devi
c8d33d06a1 add fmreader 2023-07-27 20:27:56 +02:00
devi
2089c3cc7b fix import name 2023-07-27 17:16:27 +02:00
devi
5df1da2bfd fix nswf 2023-07-27 16:52:56 +02:00
devi
70fda48983 fix nswf 2023-07-27 16:52:31 +02:00
devi
36fe5cb625 Merge branch 'KotatsuApp:master' into master 2023-07-27 16:45:45 +02:00
devi
7e2a00520b fix and add updated on template Mmrcms 2023-07-27 16:42:28 +02:00
caeeb699d1 Add content type field to sources 2023-07-27 16:33:46 +03:00
devi
f6dc4f9a00 add source 2023-07-26 19:04:43 +02:00
devi
28135aed66 add 2 new template and new source 2023-07-25 21:30:07 +02:00
8e452f4271 [MangaLib] Fix Android compatibility 2023-07-24 20:09:27 +03:00
f4656efebd Update packages structure 2023-07-24 20:02:08 +03:00
42cc0430f8 Merge pull request #220 from davvarrr/master
fix some source and add source
2023-07-24 15:54:21 +03:00
81dd8b58c8 Merge pull request #219 from VietAnh14/source/hentaivn
Add new source HentaiVN
2023-07-24 15:53:38 +03:00
415da81b1a [Komikcast] Fix chatpers names #221 2023-07-24 11:07:38 +03:00
74a20455f2 [MangaLib] Fix pages url 2023-07-24 11:06:36 +03:00
devi
0bdce39def Update src/main/kotlin/org/koitharu/kotatsu/parsers/site/vi/LxManga.kt
Co-authored-by: Koitharu <nvasya95@gmail.com>
2023-07-24 08:50:40 +02:00
devi
24f7d8b5bd Update src/main/kotlin/org/koitharu/kotatsu/parsers/site/fr/LugnicaScans.kt
Co-authored-by: Koitharu <nvasya95@gmail.com>
2023-07-24 08:50:33 +02:00
devi
18adc66f53 Update src/main/kotlin/org/koitharu/kotatsu/parsers/site/fr/LugnicaScans.kt
Co-authored-by: Koitharu <nvasya95@gmail.com>
2023-07-24 08:50:26 +02:00
devi
8c1c855531 Update src/main/kotlin/org/koitharu/kotatsu/parsers/site/fr/LugnicaScans.kt
Co-authored-by: Koitharu <nvasya95@gmail.com>
2023-07-24 08:50:17 +02:00
devi
02ae76baf2 add new set order 2023-07-23 16:23:57 +02:00
devi
b5c44383e8 add lxmanga.net 2023-07-23 14:45:46 +02:00
devi
c48ebb937b add Lugnica Scans 2023-07-23 12:29:50 +02:00
devi
49c30d1d26 Merge remote-tracking branch 'origin/master' 2023-07-23 10:16:13 +02:00
devi
cb1a7ece8c fix some source and add 3 source 2023-07-23 10:15:45 +02:00
devi
4d60ae3cb6 fix some source and add 3 source 2023-07-23 10:15:18 +02:00
vianh
fe64620e9f [HentaiVN] Load pages using api 2023-07-23 15:00:36 +07:00
vianh
a42ca9e2da [HentaiVN] New source 2023-07-23 14:06:35 +07:00
506f6f7d30 Merge pull request #217 from davvarrr/master
add source and fix
2023-07-22 13:35:22 +03:00
devi
380861f29e Group sites by language 2023-07-22 12:13:49 +02:00
devi
9afc046d7b Change logic date on mangareader
Change logic domain on mangareader
2023-07-22 11:15:18 +02:00
devi
aec87a9a67 add source and fix japscan 2023-07-22 09:38:56 +02:00
69e0a531df [Madara] Refactoring 2023-07-19 11:49:55 +03:00
c85101991f Merge pull request #214 from davvarrr/master
Add source and create decode url image
2023-07-19 11:14:00 +03:00
devi
99e87fd97a Merge remote-tracking branch 'origin/master' 2023-07-18 20:00:48 +02:00
devi
665ec6b7ed fix japscan 2023-07-18 20:00:33 +02:00
devi
01fbd209c8 Merge branch 'KotatsuApp:master' into master 2023-07-18 19:13:02 +02:00
devi
f0fa8d59a1 add source and create decode url image 2023-07-18 19:12:25 +02:00
a1598fd712 [MangaInUa] Fix parsing chapters and pages 2023-07-18 09:50:55 +03:00
db96a1ff2e [Grouple] Fix tags parsing 2023-07-17 12:20:55 +03:00
833a42e798 Merge pull request #206 from bapeey/tmo
New parser: TuMangaOnline
2023-07-17 12:20:34 +03:00
922c665ef5 Merge pull request #210 from davvarrr/master
Add new source, fix and other
2023-07-17 11:13:13 +03:00
01842e9870 Merge pull request #212 from VietAnh14/master
[BlogTruyen] Update domain, user agent
2023-07-17 11:07:38 +03:00
vianh
a235f6281c [BlogTruyen] Update domain, user agent 2023-07-17 14:52:09 +07:00
devi
c9e7d4f483 add source requested 2023-07-16 20:02:36 +02:00
devi
94496ecc02 fix Asura Scans 2023-07-16 18:51:25 +02:00
devi
5104a8b542 add source request and remove source duplicate 2023-07-16 08:31:50 +02:00
devi
e67beae235 add source request 2023-07-15 21:20:23 +02:00
devi
7c4bffede2 minor fix and add quick source ar 2023-07-15 19:41:28 +02:00
devi
8df620b1cd fix 2023-07-15 19:08:03 +02:00
devi
cbbdd13c8e add new source ( lise below )
fix multiple source. ( lise below )
update madra parser :
-> new option on getListPage for take liste without Ajax
-> add format date
-> change defaut date parterne ( the news is most used by default on sites )
2023-07-15 15:52:07 +02:00
3874fe31d5 [TuMangaOnline] Fixes 2023-07-11 10:58:51 +03:00
a8c086b340 Merge branch 'tmo' of github.com:bapeey/kotatsu-parsers into bapeey-tmo 2023-07-11 10:49:12 +03:00
06a043d290 Refactor Madara parser 2023-07-11 10:36:21 +03:00
5a15014369 Add additional methods to WebClient 2023-07-11 10:08:02 +03:00
7fe1f4b3ff Merge pull request #208 from davvarrr/master
add source and other
2023-07-11 09:56:43 +03:00
devi
0055146264 fix and quick add source request 2023-07-10 17:12:46 +02:00
seew3l
cc26897fb9 Opps 2023-07-10 09:37:35 -05:00
seew3l
46535f82af Apply requested changes 2023-07-10 09:35:26 -05:00
seew3l
23a7b945af Add headers to getPages request 2023-07-09 13:34:15 -05:00
devi
4fb6ff63b9 Merge remote-tracking branch 'origin/master' 2023-07-09 19:56:04 +02:00
devi
c2e90a210c add new source and add other 2023-07-09 19:51:53 +02:00
seew3l
22d59e46c8 Get domain from ConfigKey 2023-07-09 10:18:20 -05:00
seew3l
e462b7df97 Remove unused val 2023-07-09 10:07:25 -05:00
seew3l
af8fb952df Remove comments 2023-07-08 23:28:31 -05:00
seew3l
a3fa04bcb7 Remove comments 2023-07-08 23:27:48 -05:00
seew3l
adf6fea4ac Add TMO 2023-07-08 23:07:25 -05:00
69af4fe586 Merge pull request #203 from davvarrr/master
80 new source
2023-07-06 06:57:00 +03:00
devi
31476a55f9 Fix small bug parse on desc 2023-07-04 21:20:42 +02:00
devi
c833563907 add 10 new source 2023-07-04 18:21:46 +02:00
devi
74a4a3a8b4 And official Manga Origines 2023-07-04 16:01:10 +02:00
devi
141bca517d Add Koitharu optimization, and rename Manga origines 2023-07-04 15:49:50 +02:00
devi
b9eb6363b7 Add 5 new source 2023-07-03 19:31:59 +02:00
devi
c3bed26739 Add 3 new source "fr" 2023-07-03 18:05:30 +02:00
devi
a15d54fe11 Add 10 new source 2023-07-03 17:26:46 +02:00
devi
1998ceac5a Add 3 new sources 2023-07-02 23:01:37 +02:00
devi
4192ffdbfe Add ScantradUnion 2023-07-02 15:31:24 +02:00
devi
5ec7227886 Add ScantradUnion 2023-07-02 15:30:52 +02:00
devi
a9ebc372f6 Add 10 new source 2023-07-02 12:03:07 +02:00
devi
1e98104eca Add new source and add new scrapper for Chapter on theme madara 2023-07-01 21:02:20 +02:00
devi
d05d5fa911 Add new source 2023-07-01 18:33:05 +02:00
07df5a81cf [Chan] Add related manga implementation 2023-07-01 12:27:27 +03:00
9861a9aa6b [Grouple] Add related manga implementation 2023-07-01 12:15:22 +03:00
b65e1c498e Simple related manga search implementation 2023-07-01 11:18:10 +03:00
240562037d Merge pull request #202 from davvarrr/master
rework madara theme and fix many parser
2023-07-01 11:12:55 +03:00
b3f79e77c3 [JapScan] Fix code style 2023-07-01 11:12:24 +03:00
db3c73156e [JapScan] Fix code style 2023-07-01 11:12:15 +03:00
devi
44b3b9b0c1 Merge remote-tracking branch 'origin/master' 2023-06-30 21:22:20 +02:00
devi
ea8c1de44d rework madara theme and fix som parser 2023-06-30 21:22:00 +02:00
devi
8195d0190a remove miss test in local 2023-06-30 09:59:19 +02:00
devi
8fd10eabd1 fix japscan and fix wrong date on 2 source 2023-06-29 20:16:07 +02:00
devi
7b28072ee5 delete table mode for automatic detect mode and chage defaut value for date and list url 2023-06-28 19:59:11 +02:00
devi
b37db4d680 delete table mode for automatic detect mode and chage defaut value for date and list url 2023-06-28 19:58:48 +02:00
92bfc7e9fa [Bentomanga] Fix UserAgent 2023-06-28 13:21:55 +03:00
b3e97760db [DesuMe] Update headers 2023-06-28 12:58:37 +03:00
3895312bfc Refactor MangaReaderParser 2023-06-28 12:31:31 +03:00
06d2976bb5 Batch sources fix 2023-06-28 12:05:41 +03:00
da4566f82f Fix GitHub workflow 2023-06-28 12:01:33 +03:00
b8f0ea4690 Merge pull request #195 from davvarrr/patch-16
Create AiyuMangaScanlation.kt
2023-06-27 12:32:29 +03:00
bb5e49758a Merge pull request #194 from davvarrr/patch-15
54 new source and rework
2023-06-27 12:31:22 +03:00
devi
e617904b62 Update MangaScantrad.kt 2023-06-26 23:25:49 +02:00
devi
b0e455e163 Update ScantradVf.kt 2023-06-26 23:24:09 +02:00
devi
6c31cea155 Update HhentaiFr.kt 2023-06-26 23:23:41 +02:00
devi
9c71d1b57c Update Hentaizone.kt 2023-06-26 23:23:09 +02:00
devi
e4f12149ed fix 2023-06-26 23:22:44 +02:00
devi
f6d264f37d Update MangaScantrad.kt 2023-06-26 23:22:00 +02:00
devi
aa734e5ad3 fix 2023-06-26 23:21:24 +02:00
devi
591291e9d6 fix status 2023-06-26 22:25:35 +02:00
devi
2311a56658 add new parse language for status 2023-06-26 22:00:36 +02:00
514e4473a1 Merge branch 'master' into patch-15 2023-06-26 13:53:57 +03:00
7ccacf4c27 Fix plugins versions 2023-06-26 13:52:49 +03:00
15f226fc73 Update gradle config 2023-06-26 13:29:13 +03:00
1388196288 Update gradle config 2023-06-26 13:27:31 +03:00
fe024cb306 Merge branch 'master' into patch-15 2023-06-26 12:15:48 +03:00
04e7edaa3e Fix GitHub worklflow Java version 2023-06-26 12:11:29 +03:00
devi
87abdf6346 Update MangaReaderParser.kt 2023-06-26 08:08:36 +02:00
devi
c4fad321db Update MangaReaderParser.kt 2023-06-26 07:08:25 +02:00
devi
c17c9d9ccc Update MangaReaderParser.kt
Add long for parse onging / finished
2023-06-26 07:07:01 +02:00
devi
d27c208ab7 Create AiyuMangaScanlation.kt
add aiyumangascanlation.com "es"
2023-06-24 23:15:21 +02:00
devi
90966aa7e4 Update MangaReaderParser.kt
add 54 new source 
rework parseInfoList
Source sorting part language
2023-06-24 23:12:38 +02:00
553a6c709a Reformat code 2023-06-24 16:29:42 +03:00
16698201f4 Merge pull request #191 from davvarrr/patch-12
Create ScantradVf.kt
2023-06-24 13:49:25 +03:00
33f27a8af7 Merge pull request #190 from davvarrr/patch-11
Create MangaScantrad.kt
2023-06-24 13:49:04 +03:00
48ec877ad5 Merge pull request #189 from davvarrr/patch-10
Create Hipercool.kt
2023-06-24 13:48:44 +03:00
54002ce27f Merge pull request #188 from davvarrr/patch-9
Create Hentaiteca.kt
2023-06-24 13:45:47 +03:00
de03d80e10 Merge pull request #186 from davvarrr/patch-7
Create Atlantisscan.kt
2023-06-24 13:45:30 +03:00
60954c38b6 Merge pull request #187 from davvarrr/patch-8
Create FrScan.kt
2023-06-24 13:44:24 +03:00
ba41d5f4ce Merge pull request #184 from davvarrr/patch-5
add french parse date and new source
2023-06-24 13:44:06 +03:00
9b76f42138 Merge pull request #185 from davvarrr/patch-6
Create AstralManga.kt
2023-06-24 13:43:40 +03:00
5a0306b4b5 Merge pull request #192 from davvarrr/patch-13
Create HhentaiFr.kt
2023-06-24 13:43:08 +03:00
b70ddf0a7c Merge pull request #193 from davvarrr/patch-14
Create Hentaizone.kt
2023-06-24 13:42:47 +03:00
devi
f53174258c add Stkissmanga "en"
add Stkissmanga "en"
2023-06-23 19:33:51 +02:00
devi
36706d998e Create Hentaizone.kt 2023-06-23 18:33:44 +02:00
devi
65ec3ca5c6 Update Hipercool.kt
Add parse description
2023-06-23 08:03:58 +02:00
devi
0b0068e113 Update HhentaiFr.kt
large covers are not always present
2023-06-22 22:30:56 +02:00
devi
553777ef08 Create HhentaiFr.kt 2023-06-22 20:59:36 +02:00
devi
3cda7ffc18 Create ScantradVf.kt 2023-06-22 20:52:54 +02:00
devi
48d98548cd Create MangaScantrad.kt 2023-06-22 20:51:49 +02:00
devi
b878824d15 Create Hipercool.kt 2023-06-22 20:50:51 +02:00
devi
16a99d7b8c Create Hentaiteca.kt 2023-06-22 20:49:07 +02:00
devi
c34f250ac9 Create FrScan.kt 2023-06-22 20:48:03 +02:00
devi
e74b276c98 Create Atlantisscan.kt 2023-06-22 20:43:21 +02:00
devi
f0e61daa63 Create AstralManga.kt 2023-06-22 20:42:19 +02:00
devi
34b6f4aa32 add french parse date 2023-06-22 18:49:19 +02:00
c2b79b55f8 Update dependencies 2023-06-22 14:28:47 +03:00
875b08b1fc Merge pull request #182 from davvarrr/patch-3
Create ReaperScansFr.kt
2023-06-22 14:02:50 +03:00
0e3788d943 Merge pull request #181 from davvarrr/patch-2
Add 2 source fr on MangaReaderParser
2023-06-22 14:02:02 +03:00
396508da6f Merge pull request #183 from davvarrr/patch-4
Fix uloaddate and force Enables list
2023-06-22 14:01:03 +03:00
devi
6fc6beb067 Force Enables list
Enables list mode to be forced to load images
2023-06-21 20:03:50 +02:00
devi
939cdf71ad Fix uloaddate
allows you to use the function in madaraparser to fix dates, e.g. "2 hours ago", otherwise the app wouldn't display the chapters.
2023-06-20 21:39:54 +02:00
devi
8d7eabdd71 Create ReaperScansFr.kt
add reaperscans.fr
2023-06-20 19:05:20 +02:00
devi
d28d6b17d8 Update MangaReaderParser.kt
add epsilonscan.fr ( +18 ) "fr"
add legacy-scans.com "fr"
2023-06-20 17:34:28 +02:00
f096ca2ad3 Merge pull request #180 from drsgdfnhtyjktyjt/patch-1
Add Phenixscans.fr
2023-06-20 12:42:39 +03:00
32914eb36d Small fixes 2023-06-20 12:40:15 +03:00
sddsg
b58190ed17 Add Phenixscans.fr
I was finally able to test it at home and it works perfectly.

Could you add it, thank you.
2023-06-19 20:04:32 +02:00
86a82970fc Empty-Commit 2023-06-16 09:54:57 +03:00
9976ea5dfc [Ninemanga] Fix covers issue 2023-06-14 16:12:09 +03:00
f732582d55 [Grouple] Fix chapters parsing 2023-06-13 10:13:54 +03:00
44e28b40d3 Update issue templates 2023-06-02 10:56:20 +03:00
f969e2a8d1 [Grouple] Fix multiple genres search 2023-06-02 10:37:34 +03:00
fa7ea5b16a Fix domain config key 2023-05-27 16:42:43 +03:00
3d7a39cf67 Fix pagination 2023-05-27 12:08:24 +03:00
46f8b8e700 [Ninemanga] Remove redirect check 2023-05-24 17:01:19 +03:00
ebcc6391d6 [Mangalib] Fix nsfw detection 2023-05-23 12:52:59 +03:00
12522ce8d8 [Anibel] Fix parser 2023-05-23 12:15:12 +03:00
cae7073f87 [MangaDex][ComicK] Fix branching again 2023-05-15 15:46:59 +03:00
0366ad0e57 Fix Android compatibility 2023-05-15 11:43:46 +03:00
6b95a8c55e [MangaDex] Fix branching ##172 2023-05-15 10:56:09 +03:00
6183f6edf3 [ComicK] Fix branching #163 #167 2023-05-15 10:21:32 +03:00
83cca4ef12 [KomikTap] fix page size 2023-05-13 15:06:23 +03:00
20ad9d0472 [ALlHentai] New parser 2023-05-13 14:40:15 +03:00
96b9ac36f3 [Bentomanga] Fixes 2023-05-06 17:28:34 +03:00
65170c0fe9 [Mangadex] Fix chapters numbers 2023-05-01 09:03:11 +03:00
3e349d3db3 [Ninemanga] Fix redirects 2023-04-28 17:21:19 +03:00
b2675c3cac Fix @MangaSources annotation 2023-04-28 16:17:47 +03:00
7414cf4a4e Add @JvmField on models fields 2023-04-28 15:43:13 +03:00
f43c482539 [Honeymanga] Refactoring 2023-04-28 15:31:03 +03:00
f0fbbf4333 Merge pull request #169 from CakesTwix/master
[New source] HoneyManga
2023-04-28 15:20:30 +03:00
0d03c68c8b [Manhwaland] Fix list url 2023-04-28 15:20:00 +03:00
CakesTwix
b4a4c86049 [HoneyManga] Impl. filter by genres
and fix cover url
2023-04-26 16:35:42 +03:00
306d46ea93 [Grouple] Fix pages url extraction 2023-04-26 09:24:39 +03:00
5b94badfc2 [MangaRead] Fix list parsing 2023-04-25 08:13:32 +03:00
9adc0c5358 [Bentomanga] New parser 2023-04-24 19:34:29 +03:00
66adc65a96 [JapScan] New parser 2023-04-24 16:51:53 +03:00
8def9b5446 Improve network error handling 2023-04-23 09:20:11 +03:00
e6511061a7 [Grouple] Fix server selecting 2023-04-20 18:07:20 +03:00
CakesTwix
647a694d91 [HoneyManga] throw IllegalArgumentException if search < 3 2023-04-19 20:20:37 +03:00
CakesTwix
e5ececa42d [HoneyManga] Update code, get(), CONTENT_ENCODING 2023-04-19 19:58:25 +03:00
c321faeaa9 Merge pull request #168 from VietAnh14/source/doujindesu
[DoujinDesu] New parser
2023-04-19 19:24:29 +03:00
vianh
3112b7937c [DoujinDesu] Apply suggesions from code review 2023-04-18 23:49:46 +07:00
CakesTwix
a3565abdc3 [HoneyManga] Improve code 2023-04-18 19:35:15 +03:00
CakesTwix
34d2e15302 [New source] HoneyManga 2023-04-18 18:55:43 +03:00
1b6d1456f3 Add support for POST with json body 2023-04-17 08:35:58 +03:00
vianh
b49aee3188 [DoujinDesu] New parser 2023-04-16 22:52:22 +07:00
749f682ef9 [Madara] Fix empty tags 2023-04-15 13:22:33 +03:00
1b269efaa4 Update readme 2023-04-15 13:02:26 +03:00
2db1a8d958 Remove broken parser 2023-04-15 12:59:29 +03:00
ece7d71e09 Update domains 2023-04-15 12:51:29 +03:00
129602ea70 [IsekaiScan] Fix chapters numbers #162 2023-04-15 12:44:17 +03:00
5e3fddb652 [Grouple] Fix relative page urls 2023-04-12 18:35:46 +03:00
2340100999 Fix false nsfw flags on some madara parsers #164 2023-04-06 18:49:19 +03:00
dc0151720e [IsekaiScanEu] Fix chapters order #162 2023-04-04 07:52:24 +03:00
36e88b2c0c [Remanga] Fix chapters numbers 2023-04-04 07:40:47 +03:00
35695904d1 [Remanga] Add Accept header to image requests 2023-03-30 18:52:11 +03:00
a3ffecc00f [IsekaiScan] Fix chapters parsing 2023-03-25 10:14:35 +02:00
cc418570d5 [Remanga] Fix mirror 2023-03-25 07:32:18 +02:00
481af22ae7 [Remanga] Add mirror 2023-03-24 18:02:31 +03:00
96b9dc8b5c [ComicK] Updated API 2023-03-24 16:05:24 +03:00
93f5f70d79 Remove referrer field from page model 2023-03-19 09:28:38 +02:00
fc53b19915 Add toString to models 2023-03-18 18:20:00 +02:00
413f4a2f10 Batch parsers fixes 2023-03-09 07:32:04 +02:00
e8d299782b [Tempestfansub] Update domain #159 2023-03-05 08:09:49 +02:00
489a4ab586 [NeatManga] Fix chapters loading 2023-03-05 08:09:49 +02:00
32fa3a89e2 Merge pull request #152 from OshekharO/patch-1 2023-03-02 09:58:35 +02:00
Saksham Shekher
f2614dd3d7 Fix 2023-02-26 23:03:43 +05:30
1093584202 [Grouple] Add UserAgent config 2023-02-25 18:36:05 +02:00
f62d024342 Fix Manga District & Hentai4free parsers #157 2023-02-25 17:10:41 +02:00
eea87d8607 Fix grouple pages parsing #156 2023-02-25 16:43:06 +02:00
f4c47b5b84 Merge pull request #155 from VietAnh14/source/truyentranhlh
[TruyentranhLH]: New source
2023-02-21 18:28:27 +02:00
vianh
fa78d5dd6f [TruyentranhLH]: New source 2023-02-21 00:47:53 +07:00
779c6664be Merge pull request #154 from OshekharO/patch-2
BatoTo: New Subdomain
2023-02-18 07:18:03 +02:00
Saksham Shekher
38efd7935a BatoTo: New Subdomain 2023-02-18 07:35:24 +05:30
Saksham Shekher
03f021377d Update README.md
some minor grammatical corrections
2023-02-16 23:17:31 +05:30
cf345d2d0c [Grouple] update UserAgent 2023-02-16 19:12:35 +02:00
fb3a3f49dd Merge pull request #150 from OshekharO/patch-1
Add: KumaPoi
2023-02-16 17:59:39 +02:00
f3b731114e [DesuMe] Genres in list 2023-02-16 17:52:27 +02:00
454b24ec88 [Grouple] Selecting the fastest pages server 2023-02-16 17:52:14 +02:00
Saksham Shekher
2c6c64fbe9 Add: ToonHunter 2023-02-14 12:46:10 +05:30
Saksham Shekher
a51d6d6485 Add: Hentai20 2023-02-14 12:37:01 +05:30
Saksham Shekher
0da57c041c Fix 2023-02-14 12:09:40 +05:30
Saksham Shekher
503573f388 Add: KomikLokal 2023-02-14 12:03:18 +05:30
Saksham Shekher
29cd1f1fdf Add: KumaPoi 2023-02-14 11:54:42 +05:30
Saksham Shekher
e51b33c74a Add: Toonily, Komiklab, KomikDewasa, KomiKav (#148)
* Add: Toonily

* Add: KOMIKLAB

* Update MangaReaderParser.kt

* Add: KomikDewasa

* Add: KomiKav
2023-02-09 20:56:04 +03:00
624a3dd991 Merge pull request #146 from KotatsuApp/feature/refactor2
Improve network requests processing
2023-02-08 19:55:51 +02:00
05d705ac03 Remove deprecated "getFavicon" method 2023-02-08 11:08:57 +02:00
1e1756aa17 Refactor web client 2023-02-08 11:01:20 +02:00
91109bf6dc Merge pull request #144 from OshekharO/patch-1
Add: KomikManga & Fixes
2023-02-08 10:50:18 +02:00
Saksham Shekher
7f852e793a Add: BakaMan
Thai Manga Provider
2023-02-08 14:16:15 +05:30
Saksham Shekher
aae3747925 Add: KomikIndo
Closes #145
2023-02-08 08:27:17 +05:30
Saksham Shekher
5e23d52e8a refactor: HachiManga 2023-02-07 10:40:03 +05:30
Saksham Shekher
3bc8013d2f Merge branch 'KotatsuApp:master' into patch-1 2023-02-07 10:20:55 +05:30
00abaea324 [Remanga] Fix headers 2023-02-06 18:57:00 +02:00
d14168e325 Make headers accessible 2023-02-06 18:54:50 +02:00
c3b1556816 Add headers to CloudFlareException 2023-02-06 17:50:44 +02:00
80a2a10e71 [Remanga] Fix headers 2023-02-06 17:30:17 +02:00
Saksham Shekher
029f554d6e Add: DragonTranslation 2023-02-06 14:37:30 +05:30
Saksham Shekher
841d8f05d9 Fix: MangaRosie, PianManga 2023-02-06 14:19:49 +05:30
Saksham Shekher
7253699529 Dead: ReadManwha 2023-02-06 14:18:04 +05:30
Saksham Shekher
5fe546c94d Dead: MangaTX OT
Removed as site is not being updated
2023-02-06 14:16:40 +05:30
Saksham Shekher
31c26f16ac Dead: IsekaiScan
Url changed to isekaiscan.to ( chapter not found in kotatsu )
2023-02-06 14:13:58 +05:30
Saksham Shekher
5eda6f96b4 Dead: ReadManhwa 2023-02-06 14:10:54 +05:30
Saksham Shekher
56a54d5eb4 Fix: NeatManga 2023-02-06 14:10:12 +05:30
Saksham Shekher
7a78a67740 Dead: MangaKik 2023-02-06 14:08:40 +05:30
Saksham Shekher
742533977f Dead: X2Manga 2023-02-06 14:06:39 +05:30
Saksham Shekher
2dedaea0ab Dead: KingManga 2023-02-06 14:05:36 +05:30
Saksham Shekher
cf7f55181f Dead: Manhwa 2023-02-06 14:03:26 +05:30
Saksham Shekher
f68f0009a2 Dead: ManhwaChill 2023-02-06 14:00:07 +05:30
Saksham Shekher
4799f54fd1 Dead: All Top Manga 2023-02-06 13:57:30 +05:30
Saksham Shekher
06ff8df2e4 Dead: Heroxia 2023-02-06 13:52:37 +05:30
Saksham Shekher
9b78944f9b Fix: ManhwaList 2023-02-06 13:49:31 +05:30
Saksham Shekher
fc2781c83a Add: KomikManga 2023-02-06 13:43:02 +05:30
Saksham Shekher
176437531d Add: CosmicScans (#142)
* Add: CosmicScans

* Fix info

* Rem: nsfw

* Mangasu url changed 

Cert issue needs to be fixed
2023-02-05 19:44:48 +03:00
c28e2a72d5 [Remanga] Bypass blocking 2023-02-05 10:09:53 +03:00
7f630184c0 [GroupLe] Fix pages loading 2023-01-29 20:54:52 +03:00
e5a6b82853 [Grouple] Fix chapter link extraction 2023-01-24 09:11:09 +02:00
e1d22b1110 [ExHentai] Option to hide suspicious content 2023-01-24 08:57:19 +02:00
918318252e Fix coroutine cancellation handling in runCatching 2023-01-24 08:03:51 +02:00
e153463c35 [Grouple] Fix authorization recognizing 2023-01-24 07:58:56 +02:00
cf00732023 Misc parsers fixes 2023-01-05 11:26:58 +02:00
ef838068ff [MangahaTachi] Change url (closes #128) 2023-01-01 20:37:04 +03:00
c4acb9725f [Lib] Fix auth part 2 2022-12-31 22:28:46 +03:00
04e219ff69 Remove accidentally duplicated parser 2022-12-31 19:41:36 +03:00
106a85aa80 [Lib] Authorization fixes 2022-12-31 19:03:06 +03:00
ad6e074cce [Asura Scans] New parser 2022-12-29 18:12:22 +03:00
e89f06a35b [KomikTap] New parser (closes #40) (?) 2022-12-29 11:09:25 +03:00
238cc7529a [Asura Scans (tr)] New parser (closes #55) 2022-12-28 23:26:26 +03:00
fb22eb5fac [Tempest Fan Sub] New parser (closes #56) 2022-12-28 23:10:17 +03:00
9ee1c21a67 [ExHentai] Fix pagination 2022-12-26 18:23:15 +02:00
c7c142f4a0 [Remanga] Show bought chapters 2022-12-26 17:46:15 +02:00
efd0394398 [MangaTale] New parser (closes #125) 2022-12-25 12:22:13 +03:00
0091a8280a Merge pull request #122 from VietAnh14/source/heroxia
[MangaReader] Fix chapters order
2022-12-14 08:27:47 +02:00
vianh
f9b0e82473 [MangaReader] Add new source 2022-12-11 00:59:53 +07:00
vianh
63676c21a8 [MangaReader] fix chapters order 2022-12-10 14:55:43 +07:00
add70b4790 [ExHentai] Fix pagination 2022-12-09 18:15:01 +02:00
d18f665556 Merge pull request #121 from VietAnh14/source/heroxia
[MangaReadTheme] Add some new sources using MangaReader theme
2022-12-09 08:18:46 +02:00
vianh
42253e43b3 Add more mangareader theme sources, fix cache tags 2022-12-07 00:25:22 +07:00
vianh
ad500a9db6 [MangaReadTheme] Add some new sources using MangaReader theme 2022-12-06 23:17:31 +07:00
ViAnh
2ab03cb668 [Manhwa18] Fix parse upload date (#115) 2022-11-26 19:37:11 +03:00
ViAnh
b78ddc6166 [NetTruyen] Update domain (#116) 2022-11-26 19:36:41 +03:00
1e49d4095b [Grouple] Fix pages parsing 2022-11-21 09:03:13 +02:00
bf8a1f3db2 [MangaInUa] Fix #104 2022-11-04 17:30:38 +03:00
321cc7529e Merge pull request #96 from VietAnh14/source/manhwa18
[Manhwa18] New source
2022-10-24 19:35:30 +03:00
ViAnh
7907b3126c Apply suggestions from code review
Co-authored-by: Koitharu <nvasya95@gmail.com>
2022-10-24 23:29:53 +07:00
vianh
ccf9af0e51 [Manhwa18] New source 2022-10-24 23:05:13 +07:00
b38b43edc0 Merge pull request #94 from VietAnh14/master
[BlogTruyen] Fix image src attr change
2022-10-24 11:59:42 +03:00
vianh
5d13c55dd5 [BlogTruyen] Fix image src attr change 2022-10-23 20:37:40 +07:00
a1441e7ed7 [MangaDex] Fix large chapters loading error 2022-10-20 09:56:00 +03:00
a62662d116 Fix some madara parsers 2022-10-19 13:37:49 +03:00
5cb953eb86 Small fixes 2022-10-10 16:27:34 +03:00
505ffcf405 Mangalink parser #66 2022-10-10 16:07:14 +03:00
a45f9bfc58 ReaperScansID parser #83 2022-10-10 15:16:40 +03:00
b1990c7918 Fixes 2022-10-01 13:43:09 +03:00
e8b801684f [MangaDex] Fix error on empty description 2022-09-29 14:25:41 +03:00
de0c12c078 Add contribution guidelines 2022-09-14 20:10:18 +03:00
b3a9c5fcda Propagate deprecation from MangaParser to MangaSource 2022-09-07 13:57:00 +03:00
c73d34b04f [Madara] Fix date parsing 2022-09-07 13:45:15 +03:00
583b24854f [Remanga] Omit paid chapters 2022-09-07 13:15:51 +03:00
1c79b29845 Merge branch 'master' of github.com:KotatsuApp/kotatsu-parsers 2022-09-07 13:07:02 +03:00
c348d68983 Add deprecation annotation to some sources 2022-09-04 17:53:29 +03:00
6d7506cc1b [CloneManga] Add source 2022-09-04 17:52:54 +03:00
938 changed files with 37870 additions and 4281 deletions

View File

@@ -6,8 +6,8 @@ end_of_line = lf
indent_style = tab
max_line_length = 120
tab_width = 4
insert_final_newline = false
disabled_rules=no-wildcard-imports,no-unused-imports
insert_final_newline = true
disabled_rules = no-wildcard-imports, no-unused-imports
[{*.kt,*.kts}]
ij_kotlin_allow_trailing_comma = true

View File

@@ -1,65 +1,65 @@
name: 🐞 Issue report
description: Report a source issue in Kotatsu
labels: [bug]
description: Report a source issue with a source
labels: [ bug ]
body:
- type: input
id: source
attributes:
label: Source information
description: |
You can find the source name in navigation drawer.
placeholder: |
Example: "MangaDex"
validations:
required: true
- type: input
id: source
attributes:
label: Source information
description: |
You can find the source name in navigation drawer.
placeholder: |
Example: "MangaDex"
validations:
required: true
- type: textarea
id: reproduce-steps
attributes:
label: Steps to reproduce
description: Provide an example of the issue.
placeholder: |
Example:
1. First step
2. Second step
3. Issue here
validations:
required: false
- type: textarea
id: reproduce-steps
attributes:
label: Steps to reproduce
description: Provide an example of the issue.
placeholder: |
Example:
1. First step
2. Second step
3. Issue here
validations:
required: false
- type: input
id: kotatsu-version
attributes:
label: Kotatsu version
description: |
You can find your Kotatsu version in **Settings → About**.
placeholder: |
Example: "3.3"
validations:
required: true
- type: input
id: kotatsu-version
attributes:
label: Kotatsu version
description: |
You can find your Kotatsu version in **Settings → About**.
placeholder: |
Example: "3.3"
validations:
required: true
- type: input
id: android-version
attributes:
label: Android version
description: |
You can find this somewhere in your Android settings.
placeholder: |
Example: "Android 12"
validations:
required: false
- type: input
id: android-version
attributes:
label: Android version
description: |
You can find this somewhere in your Android settings.
placeholder: |
Example: "Android 12"
validations:
required: false
- type: textarea
id: other-details
attributes:
label: Other details
placeholder: |
Additional details and attachments.
- type: textarea
id: other-details
attributes:
label: Other details
placeholder: |
Additional details and attachments.
- type: checkboxes
id: acknowledgements
attributes:
label: Acknowledgements
options:
- label: I have searched the existing issues and this is a new ticket, **NOT** a duplicate or related to another open issue.
required: true
- type: checkboxes
id: acknowledgements
attributes:
label: Acknowledgements
options:
- label: I have searched the existing issues and this is a new ticket, **NOT** a duplicate or related to another open issue.
required: true

View File

@@ -1,50 +1,53 @@
name: 🌐 Source request
description: Suggest a new source for Kotatsu
labels: [source request]
labels: [ source request ]
body:
- type: input
id: name
attributes:
label: Source name
placeholder: |
Example: "Example Scans"
validations:
required: true
- type: markdown
attributes:
value: Please specify source **name** and **language** in the issue title
- type: input
id: name
attributes:
label: Source name
placeholder: |
Example: "Example Scans"
validations:
required: true
- type: input
id: link
attributes:
label: Source link
placeholder: |
Example: "https://example.org"
validations:
required: true
- type: input
id: link
attributes:
label: Source link
placeholder: |
Example: "https://example.org"
validations:
required: true
- type: input
id: language
attributes:
label: Language
placeholder: |
Example: "English"
validations:
required: true
- type: input
id: language
attributes:
label: Language
placeholder: |
Example: "English"
validations:
required: true
- type: textarea
id: other-details
attributes:
label: Other details
placeholder: |
Additional details and attachments.
- type: textarea
id: other-details
attributes:
label: Other details
placeholder: |
Additional details and attachments.
- type: checkboxes
id: acknowledgements
attributes:
label: Acknowledgements
options:
- label: I have searched the existing issues and this is a new ticket, **NOT** a duplicate or related to another open issue.
required: true
- label: I have checked that the source does not already exist on the app.
required: true
- label: I have checked that the source does not already exist by searching the [GitHub repository](https://github.com/KotatsuApp/kotatsu-parsers) and verified it does not appear in the code base.
required: true
- type: checkboxes
id: acknowledgements
attributes:
label: Acknowledgements
options:
- label: I have searched the existing issues and this is a new ticket, **NOT** a duplicate or related to another open issue.
required: true
- label: I have checked that the source does not already exist on the app.
required: true
- label: I have checked that the source does not already exist by searching the [GitHub repository](https://github.com/KotatsuApp/kotatsu-parsers) and verified it does not appear in the code base.
required: true

View File

@@ -1,27 +1,22 @@
name: Parsers test
on:
workflow_dispatch:
pull_request:
paths:
- 'src/main/kotlin/org/koitharu/kotatsu/parsers/site/*'
workflow_dispatch:
pull_request:
paths:
- 'src/main/kotlin/org/koitharu/kotatsu/parsers/**'
permissions:
contents: read
contents: read
jobs:
build-and-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-java@v3
with:
java-version: '11'
distribution: 'temurin'
cache: 'gradle'
- run: ./gradlew :test --tests "org.koitharu.kotatsu.parsers.MangaParserTest" || true
- run: ./gradlew generateTestsReport
- uses: actions/upload-artifact@v3
with:
name: Report
path: build/test-results-html/TEST-org.koitharu.kotatsu.parsers.MangaParserTest.htm
build-and-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-java@v3
with:
java-version: '11'
distribution: 'temurin'
cache: 'gradle'
- run: ./gradlew assemble

4
.gitignore vendored
View File

@@ -4,6 +4,7 @@
.idea/**/usage.statistics.xml
.idea/**/dictionaries
.idea/**/shelf
.idea/**/copilot
# Generated files
.idea/**/contentModel.xml
@@ -29,6 +30,7 @@
.idea/compiler.xml
.idea/jarRepositories.xml
.idea/modules.xml
.idea/ktlint-plugin.xml
.idea/*.iml
.idea/modules
*.iml
@@ -76,4 +78,4 @@ build/
.idea/codeStyles/
src/test/resources/cookies.txt
local.properties
local.properties

2
.idea/kotlinc.xml generated
View File

@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="KotlinJpsPluginSettings">
<option name="version" value="1.6.21" />
<option name="version" value="1.9.22" />
</component>
</project>

91
CONTRIBUTING.md Normal file
View File

@@ -0,0 +1,91 @@
# Contributing
The following is guide for creating a Kotatsu parsers. Thanks for taking the time to contribute!
## Prerequisites
Before you start, please note that the ability to use following technologies is **required**.
- Basic [Android development](https://developer.android.com/)
- [Kotlin](https://kotlinlang.org/)
- Web scraping ([JSoup](https://jsoup.org/)) or JSON API
### Tools
- [Android Studio](https://developer.android.com/studio)
- [IntelliJ IDEA](https://www.jetbrains.com/idea/) (Community edition is enough)
- Android device (or emulator)
Kotatsu parsers is not a part of Android application, but you can easily develop and test it directly inside an Android
application project and relocate it to the library project when done.
### Before you start
First, take a look at `kotatsu-parsers` project structure. Each parser is a single class that
extends `MangaParser` class and have a `MangaSourceParser` annotation.
Also pay attention on extensions in `util` package. For example, extensions from `Jsoup` file
should be used instead of existing JSoup functions because they have better nullability support
and improved error messages.
## Writing your parser
So, you want to create a parser, that will provide access to manga from a website.
First, you should explore a website for API availability.
If it does not contain any documentation about
API, [explore network requests](https://firefox-source-docs.mozilla.org/devtools-user/):
some websites use ajax.
- [Example](https://github.com/KotatsuApp/kotatsu-parsers/blob/master/src/main/kotlin/org/koitharu/kotatsu/parsers/site/ru/DesuMeParser.kt)
of Json API usage.
- [Example](https://github.com/KotatsuApp/kotatsu-parsers/blob/master/src/main/kotlin/org/koitharu/kotatsu/parsers/site/be/AnibelParser.kt)
of GraphQL API usage
- [Example](https://github.com/KotatsuApp/kotatsu-parsers/blob/master/src/main/kotlin/org/koitharu/kotatsu/parsers/site/en/MangaTownParser.kt)
of pure HTML parsing.
If website is based on some engine it is rationally to use common base class for this one (for example, Madara wordress
theme
and the `MadaraParser` class)
### Parser class skeleton
Parser class must have exactly one primary constructor parameter of type `MangaLoaderContext` and have an
`MangaSourceParser` annotation that provides internal name, title and language of a manga source.
All functions in `MangaParser` class are documented. Pay attention to some peculiarities:
- Never hardcode domain. Specify default domain in `configKeyDomain` field and obtain an actual one using `getDomain()`.
- All ids must be unique and domain-independent. Use `generateUid` functions with relative url or some internal id which
is unique across the manga source.
- `sortOrders` set should not be empty. If your source is not support sorting, specify one most relevance value.
- If you cannot obtain direct links to pages images inside `getPages` method, it is ok to use an intermediate url
as `Page.url` and fetch a direct link at `getPageUrl` function.
- You can use _asserts_ to check some optional fields. For example. `Manga.author` field is not required, but if your
source provide such information, add `assert(it != null)`. This will not have any effect on production but help to
find issues during unit testing.
- If your source website (or it's api) uses pages for pagination instead of offset you should extend `PagedMangaParser`
instead of `MangaParser`.
- Your parser may also implement the `Interceptor` interface for additional manipulation of all network requests and/or
responses, including image loading.
## Development process
During the development it is recommended (but not necessary) to write it directly
in the Kotatsu android application project. You can use `core.parser.DummyParser` class as a sandbox. `Dummy` manga
source is available in debug Kotatsu build.
Once parser is ready you can relocate your code into `kotatsu-parsers` library project in a `site` package and create a
Pull Request.
### Testing
It is recommended to run unit tests before submitting a PR.
- Temporary modify the `MangaSources` annotation class: specify your parser(s) name(s) and change mode
to `EnumSource.Mode.INCLUDE`
- Run the `MangaParserTest` (`gradlew :test --tests "org.koitharu.kotatsu.parsers.MangaParserTest"`)
- Optionally, you can run the `generateTestsReport` gradle task to get a pretty readable html report from test results.
## Help
If you need a help or have some questions, ask a community in our [Telegram chat](https://t.me/kotatsuapp)
or [Discord server](https://discord.gg/NNJ5RgVBC5).

View File

@@ -1,12 +1,12 @@
# Kotatsu parsers
Library that provides manga sources.
This library provides manga sources.
[![](https://jitpack.io/v/KotatsuApp/kotatsu-parsers.svg)](https://jitpack.io/#KotatsuApp/kotatsu-parsers) ![Kotlin](https://img.shields.io/github/languages/top/KotatsuApp/kotatsu-parsers) ![License](https://img.shields.io/github/license/KotatsuApp/Kotatsu) [![Discord](https://img.shields.io/discord/898363402467045416?color=5865f2&label=discord)](https://discord.gg/NNJ5RgVBC5)
[![](https://jitpack.io/v/KotatsuApp/kotatsu-parsers.svg)](https://jitpack.io/#KotatsuApp/kotatsu-parsers) ![Kotlin](https://img.shields.io/github/languages/top/KotatsuApp/kotatsu-parsers) ![License](https://img.shields.io/github/license/KotatsuApp/Kotatsu) [![Telegram](https://img.shields.io/badge/chat-telegram-60ACFF)](https://t.me/kotatsuapp) [![Discord](https://img.shields.io/discord/898363402467045416?color=5865f2&label=discord)](https://discord.gg/NNJ5RgVBC5)
### Usage
## Usage
1. Add it in your root build.gradle at the end of repositories:
1. Add it to your root build.gradle at the end of repositories:
```groovy
allprojects {
@@ -35,17 +35,33 @@ Library that provides manga sources.
}
```
See for versions at [JitPack](https://jitpack.io/#KotatsuApp/kotatsu-parsers)
Versions are available on [JitPack](https://jitpack.io/#KotatsuApp/kotatsu-parsers)
When used in Android
projects, [core library desugaring](https://developer.android.com/studio/write/java8-support#library-desugaring) with
the [NIO specification](https://developer.android.com/studio/write/java11-nio-support-table) should be enabled to
support Java 8+ features.
3. Usage in code
```kotlin
val parser = MangaSource.MANGADEX.newParser(mangaLoaderContext)
val parser = mangaLoaderContext.newParserInstance(MangaSource.MANGADEX)
```
`mangaLoaderContext` is an implementation of the `MangaLoaderContext` class.
See [Android](https://github.com/KotatsuApp/Kotatsu/blob/devel/app/src/main/java/org/koitharu/kotatsu/core/parser/MangaLoaderContextImpl.kt)
and [Non-Android](https://github.com/KotatsuApp/kotatsu-dl/blob/master/src/main/kotlin/org/koitharu/kotatsu_dl/env/MangaLoaderContextImpl.kt)
implementation examples.
See examples
of [Android](https://github.com/KotatsuApp/Kotatsu/blob/devel/app/src/main/kotlin/org/koitharu/kotatsu/core/parser/MangaLoaderContextImpl.kt)
and [Non-Android](https://github.com/KotatsuApp/kotatsu-dl/blob/master/src/jvmMain/kotlin/org/koitharu/kotatsu_dl/logic/MangaLoaderContextImpl.kt)
implementation.
Note that the `MangaSource.LOCAL` and `MangaSource.DUMMY` parsers cannot be instantiated.
Note that the `MangaSource.LOCAL` and `MangaSource.DUMMY` parsers cannot be instantiated.
## Contribution
See [CONTRIBUTING.md](./CONTRIBUTING.md) for the guidelines.
## DMCA disclaimer
The developers of this application have no affiliation with the content available in the app. It is collected from
sources freely available through any web browser.

View File

@@ -2,8 +2,8 @@ import tasks.ReportGenerateTask
plugins {
id 'java-library'
id 'org.jetbrains.kotlin.jvm'
id 'com.google.devtools.ksp'
id 'org.jetbrains.kotlin.jvm' version '1.9.22'
id 'com.google.devtools.ksp' version '1.9.22-1.0.17'
id 'maven-publish'
}
@@ -16,28 +16,27 @@ test {
compileKotlin {
kotlinOptions {
jvmTarget = '1.8'
freeCompilerArgs += [
'-opt-in=kotlin.RequiresOptIn',
'-opt-in=kotlin.contracts.ExperimentalContracts',
'-opt-in=kotlinx.coroutines.ExperimentalCoroutinesApi',
'-opt-in=org.koitharu.kotatsu.parsers.InternalParsersApi',
'-opt-in=kotlin.RequiresOptIn',
'-opt-in=kotlin.contracts.ExperimentalContracts',
'-opt-in=kotlinx.coroutines.ExperimentalCoroutinesApi',
'-opt-in=org.koitharu.kotatsu.parsers.InternalParsersApi',
]
}
}
compileTestKotlin {
kotlinOptions {
jvmTarget = '1.8'
freeCompilerArgs += [
'-opt-in=kotlin.RequiresOptIn',
'-opt-in=kotlinx.coroutines.ExperimentalCoroutinesApi',
'-opt-in=org.koitharu.kotatsu.parsers.InternalParsersApi',
'-opt-in=kotlin.RequiresOptIn',
'-opt-in=kotlinx.coroutines.ExperimentalCoroutinesApi',
'-opt-in=org.koitharu.kotatsu.parsers.InternalParsersApi',
]
}
}
kotlin {
jvmToolchain(8)
sourceSets {
main.kotlin.srcDirs += 'build/generated/ksp/main/kotlin'
}
@@ -54,20 +53,20 @@ afterEvaluate {
}
dependencies {
implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-core:1.6.4'
implementation 'com.squareup.okhttp3:okhttp:4.10.0'
implementation 'com.squareup.okio:okio:3.2.0'
api 'org.jsoup:jsoup:1.15.2'
implementation 'org.json:json:20220320'
implementation 'androidx.collection:collection-ktx:1.2.0'
implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-core:1.7.3'
implementation 'com.squareup.okhttp3:okhttp:4.12.0'
implementation 'com.squareup.okio:okio:3.7.0'
api 'org.jsoup:jsoup:1.17.2'
implementation 'org.json:json:20231013'
implementation 'androidx.collection:collection:1.4.0'
ksp project(':kotatsu-parsers-ksp')
testImplementation 'org.junit.jupiter:junit-jupiter-api:5.9.0'
testImplementation 'org.junit.jupiter:junit-jupiter-engine:5.9.0'
testImplementation 'org.junit.jupiter:junit-jupiter-params:5.9.0'
testImplementation 'org.jetbrains.kotlinx:kotlinx-coroutines-test:1.6.4'
testImplementation 'org.junit.jupiter:junit-jupiter-api:5.10.1'
testImplementation 'org.junit.jupiter:junit-jupiter-engine:5.10.1'
testImplementation 'org.junit.jupiter:junit-jupiter-params:5.10.1'
testImplementation 'org.jetbrains.kotlinx:kotlinx-coroutines-test:1.7.3'
testImplementation 'io.webfolder:quickjs:1.1.0'
}
task generateTestsReport(type: ReportGenerateTask)
tasks.register('generateTestsReport', ReportGenerateTask)

View File

@@ -1,13 +1,18 @@
plugins {
id('org.jetbrains.kotlin.jvm') version '1.6.21'
id 'org.jetbrains.kotlin.jvm' version '1.9.22'
}
repositories {
mavenCentral()
}
kotlin {
jvmToolchain(8)
}
dependencies {
implementation gradleApi()
implementation 'org.simpleframework:simple-xml:2.7.1'
implementation 'com.soywiz.korlibs.korte:korte-jvm:3.0.0-Beta5'
implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-core:1.6.3'
}
implementation 'com.soywiz.korlibs.korte:korte-jvm:4.0.10'
implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-core:1.7.3'
}

View File

@@ -1,6 +1,6 @@
package tasks
import com.soywiz.korte.Template
import korlibs.template.Template
import kotlinx.coroutines.runBlocking
import org.gradle.api.DefaultTask
import org.gradle.api.tasks.TaskAction
@@ -42,10 +42,13 @@ open class ReportGenerateTask : DefaultTask() {
val results = LinkedHashMap<String, LinkedHashMap<String, TestCase>>()
val tests = LinkedHashSet<String>()
for (case in testSuite.testCases) {
if (!case.isValid()) {
continue
}
tests.add(case.testName)
val map = results.getOrPut(case.source) { LinkedHashMap() }
val oldValue = map.put(case.testName, case)
check(oldValue == null)
check(oldValue == null) { "Check failed: $oldValue" }
}
val failPercent = (testSuite.failures.toDouble() / testSuite.tests * 100.0).roundToInt()

View File

@@ -19,14 +19,16 @@ class TestCase {
var failure: Failure? = null
val index by lazy {
name.split('|')[0].toInt()
name.split('|').getOrNull(0)?.toIntOrNull() ?: 0
}
val testName by lazy {
name.split('|')[1]
name.split('|').getOrNull(1).orEmpty()
}
val source by lazy {
name.split('|')[2]
name.split('|').getOrNull(2).orEmpty()
}
}
fun isValid() = name.count { it == '|' } == 2
}

View File

@@ -2,13 +2,15 @@
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta content="width=device-width, initial-scale=1" name="viewport">
<title>{{ testSuite.name }}</title>
<link href="https://cdn.jsdelivr.net/npm/bootstrap@5.2.0-beta1/dist/css/bootstrap.min.css" rel="stylesheet"
integrity="sha384-0evHe/X+R7YkIZDRvuzKMRqM+OrBnVFBL6DOitfPri4tjfHxaWutUpFmBp4vmVor" crossorigin="anonymous">
<script src="https://cdn.jsdelivr.net/npm/bootstrap@5.2.0-beta1/dist/js/bootstrap.bundle.min.js"
integrity="sha384-pprn3073KE6tl6bjs2QrFaJGz5/SUsLqktiwsUTF55Jfv3qYSDhgCecCxMW52nD2"
crossorigin="anonymous"></script>
<!-- CSS only -->
<link crossorigin="anonymous" href="https://cdn.jsdelivr.net/npm/bootstrap@5.2.2/dist/css/bootstrap.min.css"
integrity="sha384-Zenh87qX5JnK2Jl0vWa8Ck2rdkQ2Bzep5IDxbcnCeuOxjzrPF/et3URy9Bv1WTRi" rel="stylesheet">
<!-- JavaScript Bundle with Popper -->
<script crossorigin="anonymous"
integrity="sha384-OERcA2EqjJCMA+/3y+gxIOqMEjwtxJY7qPCqsdltbNJuaOe923+mo//f6V8Qbsw3"
src="https://cdn.jsdelivr.net/npm/bootstrap@5.2.2/dist/js/bootstrap.bundle.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/feather-icons/dist/feather.min.js"></script>
</head>
<body class="py-4">
@@ -25,67 +27,80 @@
{{ testSuite.errors }} ({{ error_percent }}%)
</div>
</div>
<div class="table-responsive mt-4">
<table class="table table-hover">
<thead>
<tr>
<th scope="col">Source</th>
{% for test in tests %}
<th scope="col" style="min-width: 5em;">{{ test }}</th>
{% endfor %}
</tr>
</thead>
{% for name, cases in results %}
<tr>
<th scope="row">{{ name }}</th>
{% for test in tests %}
{% set case = cases[test] %}
{% if case.failure == null %}
<td class="table-success text-center">
<i data-feather="check"></i>
</td>
{% else %}
{% if case.failure.type == 'java.lang.AssertionError' %}
<td class="table-warning text-center" style="cursor: pointer;"
data-bs-toggle="modal" data-bs-target="#failure_{{ case.hashCode }}">
<i data-feather="alert-triangle"></i>
</td>
{% else %}
<td class="table-danger text-center" style="cursor: pointer;"
data-bs-toggle="modal" data-bs-target="#failure_{{ case.hashCode }}">
<i data-feather="x"></i>
</td>
{% endif %}
<!--suppress HtmlUnknownTag -->
<div class="modal fade" id="failure_{{ case.hashCode }}" tabindex="-1">
<div class="modal-dialog modal-lg modal-dialog-scrollable">
<div class="modal-content">
<div class="modal-header">
<h5 class="modal-title">{{ case.testName }} failed</h5>
<button type="button" class="btn-close" data-bs-dismiss="modal"
aria-label="Close"></button>
</div>
<div class="modal-body font-monospace lh-sm bg-light" style="font-size: 0.7em;">
{{ case.failure.textHtml()|raw }}
</div>
<div class="modal-footer">
<button type="button" class="btn btn-secondary" data-bs-dismiss="modal">Close</button>
</div>
<table class="table table-hover">
<thead class="sticky-top bg-body">
<tr>
<th scope="col">Source</th>
{% for test in tests %}
<th class="text-center" scope="col" style="min-width: 5em;">{{ test }}</th>
{% endfor %}
</tr>
</thead>
{% for name, cases in results %}
<tr>
<th scope="row">{{ name }}</th>
{% for test in tests %}
{% set case = cases[test] %}
{% if case.failure == null %}
<td class="table-success text-center">
<i data-feather="check"></i>
</td>
{% else %}
{% if case.failure.type == 'java.lang.AssertionError' %}
<td class="table-warning text-center" data-bs-target="#failure_{{ case.hashCode }}"
data-bs-toggle="modal" style="cursor: pointer;">
<i data-feather="alert-triangle"></i>
</td>
{% elseif case.failure.type == 'java.net.SocketTimeoutException' or case.failure.type ==
'java.net.UnknownHostException' %}
<td class="table-secondary text-center" data-bs-target="#failure_{{ case.hashCode }}"
data-bs-toggle="modal" style="cursor: pointer;">
<i data-feather="power"></i>
</td>
{% elseif case.failure.type == 'org.koitharu.kotatsu.parsers.CloudFlareProtectedException' %}
<td class="table-secondary text-center" data-bs-target="#failure_{{ case.hashCode }}"
data-bs-toggle="modal" style="cursor: pointer;">
<i data-feather="shield"></i>
</td>
{% elseif case.failure.type == 'org.koitharu.kotatsu.parsers.exception.AuthRequiredException' %}
<td class="table-secondary text-center" data-bs-target="#failure_{{ case.hashCode }}"
data-bs-toggle="modal" style="cursor: pointer;">
<i data-feather="user-x"></i>
</td>
{% else %}
<td class="table-danger text-center" data-bs-target="#failure_{{ case.hashCode }}"
data-bs-toggle="modal" style="cursor: pointer;">
<i data-feather="x"></i>
</td>
{% endif %}
<!--suppress HtmlUnknownTag -->
<div class="modal fade" id="failure_{{ case.hashCode }}" tabindex="-1">
<div class="modal-dialog modal-lg modal-dialog-scrollable">
<div class="modal-content">
<div class="modal-header">
<h5 class="modal-title">{{ case.testName }} failed</h5>
<button aria-label="Close" class="btn-close" data-bs-dismiss="modal"
type="button"></button>
</div>
<div class="modal-body font-monospace lh-sm bg-light" style="font-size: 0.7em;">
{{ case.failure.textHtml()|raw }}
</div>
<div class="modal-footer">
<button class="btn btn-secondary" data-bs-dismiss="modal" type="button">Close</button>
</div>
</div>
</div>
{% endif %}
{% endfor %}
</tr>
</div>
{% endif %}
{% endfor %}
</table>
</div>
</tr>
{% endfor %}
</table>
</div>
<script>
feather.replace()
</script>
</body>
</html>
</html>

View File

@@ -1 +1,2 @@
kotlin.code.style=official
org.gradle.jvmargs=-Xmx4g -XX:MaxMetaspaceSize=4096m

View File

@@ -1,5 +1,5 @@
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-7.3.3-bin.zip
distributionUrl=https\://services.gradle.org/distributions/gradle-8.2-bin.zip
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists

View File

@@ -2,6 +2,10 @@ plugins {
id 'org.jetbrains.kotlin.jvm'
}
kotlin {
jvmToolchain(8)
}
dependencies {
implementation 'com.google.devtools.ksp:symbol-processing-api:1.6.21-1.0.5'
}
implementation 'com.google.devtools.ksp:symbol-processing-api:1.9.22-1.0.17'
}

View File

@@ -69,10 +69,14 @@ class ParserProcessor(
import org.koitharu.kotatsu.parsers.model.MangaSource
@Suppress("DEPRECATION")
@InternalParsersApi
@Deprecated("", replaceWith = ReplaceWith("context.newParserInstance(this)"))
fun MangaSource.newParser(context: MangaLoaderContext): MangaParser = when (this) {
""".trimIndent(),
)
//language=kotlin
sourcesWriter?.write(
"""
package org.koitharu.kotatsu.parsers.model
@@ -80,8 +84,9 @@ class ParserProcessor(
enum class MangaSource(
val title: String,
val locale: String?,
val contentType: ContentType,
) {
LOCAL("Local", null),
LOCAL("Local", null, ContentType.OTHER),
""".trimIndent(),
)
@@ -104,7 +109,7 @@ class ParserProcessor(
)
sourcesWriter?.write(
"""
DUMMY("Dummy", null),
DUMMY("Dummy", null, ContentType.OTHER),
;
}
""".trimIndent(),
@@ -116,14 +121,18 @@ class ParserProcessor(
private val factoryWriter: Writer?,
) : KSVisitorVoid() {
private val titles = HashMap<String, String>()
override fun visitClassDeclaration(classDeclaration: KSClassDeclaration, data: Unit) {
if (classDeclaration.classKind != ClassKind.CLASS || classDeclaration.isAbstract()) {
logger.error("Only non-abstract can be annotated with @MangaSourceParser", classDeclaration)
}
val annotation = classDeclaration.annotations.single { it.shortName.asString() == "MangaSourceParser" }
val deprecation = classDeclaration.annotations.singleOrNull { it.shortName.asString() == "Deprecated" }
val name = annotation.arguments.single { it.name?.asString() == "name" }.value as String
val title = annotation.arguments.single { it.name?.asString() == "title" }.value as String
val locale = annotation.arguments.single { it.name?.asString() == "locale" }.value as String
val type = annotation.arguments.single { it.name?.asString() == "type" }.value
val localeString = if (locale.isEmpty()) "null" else "\"$locale\""
val localeObj = if (locale.isEmpty()) null else Locale(locale)
val localeTitle = localeObj?.getDisplayLanguage(localeObj)
@@ -142,11 +151,21 @@ class ParserProcessor(
classDeclaration,
)
}
val className = checkNotNull(classDeclaration.qualifiedName?.asString()) { "Class name is null" }
val prevTitleClass = titles.put(title, className)
if (prevTitleClass != null) {
logger.warn("Source title duplication: \"$title\" is assigned to both $prevTitleClass and $className")
}
val className = classDeclaration.qualifiedName?.asString()
factoryWriter?.write("\tMangaSource.$name -> $className(context)\n")
val deprecationString = if (deprecation != null) {
val reason = deprecation.arguments
.find { it.name?.asString() == "message" }?.value?.toString() ?: "Unknown reason"
"@Deprecated(\"$reason\") "
} else ""
val localeComment = localeTitle?.toTitleCase(localeObj)?.let { " /* $it */" }.orEmpty()
sourcesWriter?.write("\t$name(\"$title\", $localeString$localeComment),\n")
sourcesWriter?.write("\t$deprecationString$name(\"$title\", $localeString$localeComment, ContentType.$type),\n")
}
}
}
}

View File

@@ -1,19 +1,16 @@
pluginManagement {
plugins {
id 'com.google.devtools.ksp' version '1.6.21-1.0.5'
id 'org.jetbrains.kotlin.jvm' version '1.6.21'
}
repositories {
gradlePluginPortal()
google()
mavenCentral()
gradlePluginPortal()
}
}
dependencyResolutionManagement {
repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS)
repositories {
google()
mavenCentral()
gradlePluginPortal()
}
}

View File

@@ -0,0 +1,14 @@
package org.koitharu.kotatsu.parsers
object ErrorMessages {
const val FILTER_MULTIPLE_STATES_NOT_SUPPORTED = "Multiple states are not supported by this source"
const val FILTER_MULTIPLE_GENRES_NOT_SUPPORTED = "Multiple genres are not supported by this source"
const val FILTER_MULTIPLE_CONTENT_RATING_NOT_SUPPORTED =
"Multiple Content Rating are not supported by this source"
const val FILTER_BOTH_LOCALE_GENRES_NOT_SUPPORTED =
"Filtering by both genres and locale is not supported by this source"
const val FILTER_BOTH_STATES_GENRES_NOT_SUPPORTED =
"Filtering by both genres and states is not supported by this source"
const val SEARCH_NOT_SUPPORTED = "Search is not supported by this source"
}

View File

@@ -11,4 +11,4 @@ package org.koitharu.kotatsu.parsers
@SinceKotlin("1.3")
@RequiresOptIn
@MustBeDocumented
annotation class InternalParsersApi()
annotation class InternalParsersApi

View File

@@ -1,136 +1,19 @@
package org.koitharu.kotatsu.parsers
import okhttp3.*
import okhttp3.HttpUrl.Companion.toHttpUrl
import okhttp3.MediaType.Companion.toMediaType
import okhttp3.RequestBody.Companion.toRequestBody
import org.json.JSONObject
import org.jsoup.HttpStatusException
import okhttp3.CookieJar
import okhttp3.OkHttpClient
import org.koitharu.kotatsu.parsers.config.MangaSourceConfig
import org.koitharu.kotatsu.parsers.exception.GraphQLException
import org.koitharu.kotatsu.parsers.exception.NotFoundException
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.util.await
import org.koitharu.kotatsu.parsers.util.parseJson
import java.util.*
abstract class MangaLoaderContext {
protected abstract val httpClient: OkHttpClient
abstract val httpClient: OkHttpClient
abstract val cookieJar: CookieJar
/**
* Do a GET http request to specific url
* @param url
* @param headers an additional headers for request, may be null
*/
suspend fun httpGet(url: HttpUrl, headers: Headers? = null): Response {
val request = Request.Builder()
.get()
.url(url)
if (headers != null) {
request.headers(headers)
}
return httpClient.newCall(request.build()).await().ensureSuccess()
}
suspend fun httpGet(url: String, headers: Headers? = null): Response {
return httpGet(url.toHttpUrl(), headers)
}
/**
* Do a HEAD http request to specific url
* @param url
* @param headers an additional headers for request, may be null
*/
suspend fun httpHead(url: String, headers: Headers? = null): Response {
val request = Request.Builder()
.head()
.url(url)
if (headers != null) {
request.headers(headers)
}
return httpClient.newCall(request.build()).await().ensureSuccess()
}
/**
* Do a POST http request to specific url with `multipart/form-data` payload
* @param url
* @param form payload as key=>value map
* @param headers an additional headers for request, may be null
*/
suspend fun httpPost(
url: String,
form: Map<String, String>,
headers: Headers? = null,
): Response {
val body = FormBody.Builder()
form.forEach { (k, v) ->
body.addEncoded(k, v)
}
val request = Request.Builder()
.post(body.build())
.url(url)
if (headers != null) {
request.headers(headers)
}
return httpClient.newCall(request.build()).await().ensureSuccess()
}
/**
* Do a POST http request to specific url with `multipart/form-data` payload
* @param url
* @param payload payload as `key=value` string with `&` separator
* @param headers an additional headers for request, may be null
*/
suspend fun httpPost(
url: String,
payload: String,
headers: Headers?,
): Response {
val body = FormBody.Builder()
payload.split('&').forEach {
val pos = it.indexOf('=')
if (pos != -1) {
val k = it.substring(0, pos)
val v = it.substring(pos + 1)
body.addEncoded(k, v)
}
}
val request = Request.Builder()
.post(body.build())
.url(url)
if (headers != null) {
request.headers(headers)
}
return httpClient.newCall(request.build()).await().ensureSuccess()
}
/**
* Do a GraphQL request to specific url
* @param endpoint an url
* @param query GraphQL request payload
*/
suspend fun graphQLQuery(endpoint: String, query: String): JSONObject {
val body = JSONObject()
body.put("operationName", null as Any?)
body.put("variables", JSONObject())
body.put("query", "{$query}")
val mediaType = "application/json; charset=utf-8".toMediaType()
val requestBody = body.toString().toRequestBody(mediaType)
val request = Request.Builder()
.post(requestBody)
.url(endpoint)
val json = httpClient.newCall(request.build()).await().ensureSuccess().parseJson()
json.optJSONArray("errors")?.let {
if (it.length() != 0) {
throw GraphQLException(it)
}
}
return json
}
@Suppress("DEPRECATION")
fun newParserInstance(source: MangaSource): MangaParser = source.newParser(this)
open fun encodeBase64(data: ByteArray): String = Base64.getEncoder().encodeToString(data)
@@ -146,21 +29,4 @@ abstract class MangaLoaderContext {
abstract suspend fun evaluateJs(script: String): String?
abstract fun getConfig(source: MangaSource): MangaSourceConfig
private fun Response.ensureSuccess(): Response {
val exception: Exception? = when (code) { // Catch some error codes, not all
404 -> NotFoundException(message, request.url.toString())
in 500..599 -> HttpStatusException(message, code, request.url.toString())
else -> null
}
if (exception != null) {
runCatching {
close()
}.onFailure {
exception.addSuppressed(it)
}
throw exception
}
return this
}
}
}

View File

@@ -3,67 +3,117 @@ package org.koitharu.kotatsu.parsers
import androidx.annotation.CallSuper
import androidx.annotation.VisibleForTesting
import okhttp3.Headers
import okhttp3.HttpUrl
import org.jsoup.nodes.Element
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.exception.ParseException
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.network.OkHttpWebClient
import org.koitharu.kotatsu.parsers.network.UserAgents
import org.koitharu.kotatsu.parsers.network.WebClient
import org.koitharu.kotatsu.parsers.util.FaviconParser
import org.koitharu.kotatsu.parsers.util.RelatedMangaFinder
import org.koitharu.kotatsu.parsers.util.domain
import org.koitharu.kotatsu.parsers.util.toAbsoluteUrl
import java.util.*
abstract class MangaParser @InternalParsersApi constructor(val source: MangaSource) {
protected abstract val context: MangaLoaderContext
abstract class MangaParser @InternalParsersApi constructor(
@property:InternalParsersApi val context: MangaLoaderContext,
val source: MangaSource,
) {
/**
* Supported [SortOrder] variants. Must not be empty.
*
* For better performance use [EnumSet] for more than one item.
*/
abstract val sortOrders: Set<SortOrder>
abstract val availableSortOrders: Set<SortOrder>
/**
* Supported [MangaState] variants for filtering. May be empty.
*
* For better performance use [EnumSet] for more than one item.
*/
open val availableStates: Set<MangaState>
get() = emptySet()
open val availableContentRating: Set<ContentRating>
get() = emptySet()
/**
* Whether parser supports filtering by more than one tag
*/
open val isMultipleTagsSupported: Boolean = true
/**
* Whether parser supports tagsExclude field in filter
*/
open val isTagsExclusionSupported: Boolean = false
/**
* Whether parser supports searching by string query using [MangaListFilter.Search]
*/
open val isSearchSupported: Boolean = true
@Deprecated(
message = "Use availableSortOrders instead",
replaceWith = ReplaceWith("availableSortOrders"),
)
open val sortOrders: Set<SortOrder>
get() = availableSortOrders
val config by lazy { context.getConfig(source) }
val sourceLocale: Locale?
get() = source.locale?.let { Locale(it) }
open val sourceLocale: Locale
get() = source.locale?.let { Locale(it) } ?: Locale.ROOT
val isNsfwSource = source.contentType == ContentType.HENTAI
/**
* Provide default domain and available alternatives, if any.
*
* Never hardcode domain in requests, use [getDomain] instead.
* Never hardcode domain in requests, use [domain] instead.
*/
protected abstract val configKeyDomain: ConfigKey.Domain
@InternalParsersApi
abstract val configKeyDomain: ConfigKey.Domain
@VisibleForTesting(otherwise = VisibleForTesting.PROTECTED)
internal open val headers: Headers? = null
open val headers: Headers = Headers.Builder()
.add("User-Agent", UserAgents.CHROME_MOBILE)
.build()
/**
* Used as fallback if value of `sortOrder` passed to [getList] is null
*/
protected open val defaultSortOrder: SortOrder
@VisibleForTesting(otherwise = VisibleForTesting.PROTECTED)
open val defaultSortOrder: SortOrder
get() {
val supported = sortOrders
return SortOrder.values().first { it in supported }
val supported = availableSortOrders
return SortOrder.entries.first { it in supported }
}
@JvmField
protected val webClient: WebClient = OkHttpWebClient(context.httpClient, source)
/**
* Parse list of manga by specified criteria
*
* @param offset starting from 0 and used for pagination.
* Note than passed value may not be divisible by internal page size, so you should adjust it manually.
* @param query search query, may be null or empty if no search needed
* @param tags genres for filtering, values from [getTags] and [Manga.tags]. May be null or empty
* @param sortOrder one of [sortOrders] or null for default value
* @param tags genres for filtering, values from [getAvailableTags] and [Manga.tags]. May be null or empty
* @param sortOrder one of [availableSortOrders] or null for default value
*/
@JvmSynthetic
@InternalParsersApi
abstract suspend fun getList(
@Deprecated(
"Use getList with filter instead",
replaceWith = ReplaceWith("getList(offset, filter)"),
)
open suspend fun getList(
offset: Int,
query: String?,
tags: Set<MangaTag>?,
tagsExclude: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga>
): List<Manga> = throw NotImplementedError("Please implement getList(offset, filter) instead")
/**
* Parse list of manga with search by text query
@@ -71,8 +121,15 @@ abstract class MangaParser @InternalParsersApi constructor(val source: MangaSour
* @param offset starting from 0 and used for pagination.
* @param query search query
*/
@Deprecated(
"Use getList with filter instead",
ReplaceWith(
"getList(offset, MangaListFilter.Search(query))",
"org.koitharu.kotatsu.parsers.model.MangaListFilter",
),
)
open suspend fun getList(offset: Int, query: String): List<Manga> {
return getList(offset, query, null, defaultSortOrder)
return getList(offset, MangaListFilter.Search(query))
}
/**
@@ -80,11 +137,62 @@ abstract class MangaParser @InternalParsersApi constructor(val source: MangaSour
*
* @param offset starting from 0 and used for pagination.
* Note than passed value may not be divisible by internal page size, so you should adjust it manually.
* @param tags genres for filtering, values from [getTags] and [Manga.tags]. May be null or empty
* @param sortOrder one of [sortOrders] or null for default value
* @param tags genres for filtering, values from [getAvailableTags] and [Manga.tags]. May be null or empty
* @param sortOrder one of [availableSortOrders] or null for default value
*/
open suspend fun getList(offset: Int, tags: Set<MangaTag>?, sortOrder: SortOrder?): List<Manga> {
return getList(offset, null, tags, sortOrder ?: defaultSortOrder)
@Deprecated(
"Use getList with filter instead",
ReplaceWith(
"getList(offset, MangaListFilter.Advanced(sortOrder, tags, null, emptySet()))",
"org.koitharu.kotatsu.parsers.model.MangaListFilter",
),
)
open suspend fun getList(
offset: Int,
tags: Set<MangaTag>?,
tagsExclude: Set<MangaTag>?,
sortOrder: SortOrder?,
): List<Manga> {
return getList(
offset,
MangaListFilter.Advanced(
sortOrder = sortOrder ?: defaultSortOrder,
tags = tags.orEmpty(),
tagsExclude = tagsExclude.orEmpty(),
locale = null,
states = emptySet(),
contentRating = emptySet(),
),
)
}
@Suppress("DEPRECATION")
open suspend fun getList(offset: Int, filter: MangaListFilter?): List<Manga> {
return when (filter) {
is MangaListFilter.Advanced -> getList(
offset = offset,
query = null,
tags = filter.tags,
tagsExclude = filter.tagsExclude,
sortOrder = filter.sortOrder,
)
is MangaListFilter.Search -> getList(
offset = offset,
query = filter.query,
tags = null,
tagsExclude = null,
sortOrder = defaultSortOrder,
)
null -> getList(
offset = offset,
query = null,
tags = null,
tagsExclude = null,
sortOrder = defaultSortOrder,
)
}
}
/**
@@ -103,27 +211,29 @@ abstract class MangaParser @InternalParsersApi constructor(val source: MangaSour
/**
* Fetch direct link to the page image.
*/
open suspend fun getPageUrl(page: MangaPage): String = page.url.toAbsoluteUrl(getDomain())
open suspend fun getPageUrl(page: MangaPage): String = page.url.toAbsoluteUrl(domain)
/**
* Fetch available tags (genres) for source
*/
abstract suspend fun getTags(): Set<MangaTag>
abstract suspend fun getAvailableTags(): Set<MangaTag>
/**
* Returns direct link to the website favicon
* Fetch available locales for multilingual sources
*/
open suspend fun getAvailableLocales(): Set<Locale> = emptySet()
@Deprecated(
message = "Use parseFavicons() to get multiple favicons with different size",
replaceWith = ReplaceWith("parseFavicons()"),
message = "Use getAvailableTags instead",
replaceWith = ReplaceWith("getAvailableTags()"),
)
open fun getFaviconUrl() = "https://${getDomain()}/favicon.ico"
suspend fun getTags(): Set<MangaTag> = getAvailableTags()
/**
* Parse favicons from the main page of the source`s website
*/
open suspend fun getFavicons(): Favicons {
return FaviconParser(context, getDomain(), headers).parseFavicons()
return FaviconParser(webClient, domain).parseFavicons()
}
@CallSuper
@@ -131,70 +241,13 @@ abstract class MangaParser @InternalParsersApi constructor(val source: MangaSour
keys.add(configKeyDomain)
}
/* Utils */
fun getDomain(): String {
return config[configKeyDomain]
open suspend fun getRelatedManga(seed: Manga): List<Manga> {
return RelatedMangaFinder(listOf(this)).invoke(seed)
}
fun getDomain(subdomain: String): String {
val domain = getDomain()
return subdomain + "." + domain.removePrefix("www.")
protected fun getParser(source: MangaSource) = if (this.source == source) {
this
} else {
context.newParserInstance(source)
}
fun urlBuilder(): HttpUrl.Builder {
return HttpUrl.Builder()
.scheme("https")
.host(getDomain())
}
/**
* Create a unique id for [Manga]/[MangaChapter]/[MangaPage].
* @param url must be relative url, without a domain
* @see [Manga.id]
* @see [MangaChapter.id]
* @see [MangaPage.id]
*/
@InternalParsersApi
protected fun generateUid(url: String): Long {
var h = 1125899906842597L
source.name.forEach { c ->
h = 31 * h + c.code
}
url.forEach { c ->
h = 31 * h + c.code
}
return h
}
/**
* Create a unique id for [Manga]/[MangaChapter]/[MangaPage].
* @param id an internal identifier
* @see [Manga.id]
* @see [MangaChapter.id]
* @see [MangaPage.id]
*/
@InternalParsersApi
protected fun generateUid(id: Long): Long {
var h = 1125899906842597L
source.name.forEach { c ->
h = 31 * h + c.code
}
h = 31 * h + id
return h
}
@InternalParsersApi
protected fun Element.parseFailed(message: String? = null): Nothing {
throw ParseException(message, ownerDocument()?.location() ?: baseUri(), null)
}
@InternalParsersApi
protected fun Set<MangaTag>?.oneOrThrowIfMany(): MangaTag? {
return when {
isNullOrEmpty() -> null
size == 1 -> first()
else -> throw IllegalArgumentException("Multiple genres are not supported by this source")
}
}
}
}

View File

@@ -1,5 +1,7 @@
package org.koitharu.kotatsu.parsers
import org.koitharu.kotatsu.parsers.model.ContentType
/**
* Annotate each [MangaParser] implementation with this annotation, used by codegen
*/
@@ -11,10 +13,15 @@ annotation class MangaSourceParser(
val name: String,
/**
* User-friendly title of manga source. In most case equals the website name.
* Avoid extra whitespaces between the words if it is not required.
*/
val title: String,
/**
* Language code (for example "en" or "ru") or blank if parser provide manga on different languages.
*/
val locale: String = "",
)
/**
* Type of content provided by parser. See [ContentType] for more info
*/
val type: ContentType = ContentType.MANGA,
)

View File

@@ -1,27 +1,33 @@
package org.koitharu.kotatsu.parsers
import org.koitharu.kotatsu.parsers.model.Manga
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.model.MangaTag
import org.koitharu.kotatsu.parsers.model.SortOrder
import androidx.annotation.RestrictTo
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.Paginator
@InternalParsersApi
abstract class PagedMangaParser(
context: MangaLoaderContext,
source: MangaSource,
pageSize: Int,
@RestrictTo(RestrictTo.Scope.TESTS) @JvmField internal val pageSize: Int,
searchPageSize: Int = pageSize,
) : MangaParser(source) {
) : MangaParser(context, source) {
@JvmField
protected val paginator = Paginator(pageSize)
@JvmField
protected val searchPaginator = Paginator(searchPageSize)
override suspend fun getList(offset: Int, query: String): List<Manga> {
return getList(searchPaginator, offset, query, null, defaultSortOrder)
}
override suspend fun getList(offset: Int, tags: Set<MangaTag>?, sortOrder: SortOrder?): List<Manga> {
return getList(paginator, offset, null, tags, sortOrder ?: defaultSortOrder)
final override suspend fun getList(offset: Int, filter: MangaListFilter?): List<Manga> {
return getList(
paginator = if (filter is MangaListFilter.Search) {
searchPaginator
} else {
paginator
},
offset = offset,
filter = filter,
)
}
@InternalParsersApi
@@ -30,21 +36,54 @@ abstract class PagedMangaParser(
offset: Int,
query: String?,
tags: Set<MangaTag>?,
tagsExclude: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> = throw UnsupportedOperationException("You should use getListPage for PagedMangaParser")
abstract suspend fun getListPage(page: Int, query: String?, tags: Set<MangaTag>?, sortOrder: SortOrder): List<Manga>
open suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
tagsExclude: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> = throw NotImplementedError("Please implement getListPage(page, filter) instead")
open suspend fun getListPage(page: Int, filter: MangaListFilter?): List<Manga> {
return when (filter) {
is MangaListFilter.Advanced -> getListPage(
page = page,
query = null,
tags = filter.tags,
tagsExclude = filter.tagsExclude,
sortOrder = filter.sortOrder,
)
is MangaListFilter.Search -> getListPage(
page = page,
query = filter.query,
tags = null,
tagsExclude = null,
sortOrder = defaultSortOrder,
)
null -> getListPage(
page = page,
query = null,
tags = null,
tagsExclude = null,
sortOrder = defaultSortOrder,
)
}
}
private suspend fun getList(
paginator: Paginator,
offset: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
filter: MangaListFilter?,
): List<Manga> {
val page = paginator.getPage(offset)
val list = getListPage(page, query, tags, sortOrder)
val list = getListPage(page, filter)
paginator.onListReceived(offset, page, list.size)
return list
}
}
}

View File

@@ -1,13 +1,32 @@
package org.koitharu.kotatsu.parsers.config
sealed class ConfigKey<T>(
val key: String,
@JvmField val key: String,
) {
abstract val defaultValue: T
class Domain(
@JvmField @JvmSuppressWildcards vararg val presetValues: String,
) : ConfigKey<String>("domain") {
init {
require(presetValues.isNotEmpty()) { "You must provide at least one domain" }
}
override val defaultValue: String
get() = presetValues.first()
}
class ShowSuspiciousContent(
override val defaultValue: Boolean,
) : ConfigKey<Boolean>("show_suspicious")
class UserAgent(
override val defaultValue: String,
val presetValues: Array<String>?,
) : ConfigKey<String>("domain")
}
) : ConfigKey<String>("user_agent")
class SplitByTranslations(
override val defaultValue: Boolean,
) : ConfigKey<Boolean>("split_translations")
}

View File

@@ -6,6 +6,7 @@ import org.koitharu.kotatsu.parsers.model.MangaSource
/**
* Authorization is required for access to the requested content
*/
class AuthRequiredException @InternalParsersApi constructor(
class AuthRequiredException @InternalParsersApi @JvmOverloads constructor(
val source: MangaSource,
) : RuntimeException("Authorization required")
cause: Throwable? = null,
) : RuntimeException("Authorization required", cause)

View File

@@ -1,7 +0,0 @@
package org.koitharu.kotatsu.parsers.exception
import okio.IOException
class CloudFlareProtectedException(
val url: String,
) : IOException("Protected by CloudFlare: $url")

View File

@@ -0,0 +1,7 @@
package org.koitharu.kotatsu.parsers.model
enum class ContentRating {
SAFE,
SUGGESTIVE,
ADULT
}

View File

@@ -0,0 +1,24 @@
package org.koitharu.kotatsu.parsers.model
enum class ContentType {
/**
* Standard manga, manhua, webtoons, etc
*/
MANGA,
/**
* Use this if the source provides mostly nsfw content.
*/
HENTAI,
/**
* Western comics
*/
COMICS,
/**
* Use this type if no other suits your needs. For example, for an indie manga
*/
OTHER,
}

View File

@@ -3,11 +3,12 @@ package org.koitharu.kotatsu.parsers.model
import okhttp3.HttpUrl.Companion.toHttpUrl
class Favicon internal constructor(
val url: String,
val size: Int,
internal val rel: String?,
@JvmField val url: String,
@JvmField val size: Int,
@JvmField internal val rel: String?,
) : Comparable<Favicon> {
@JvmField
val type: String = url.toHttpUrl().pathSegments.last()
.substringAfterLast('.', "").lowercase()
@@ -48,4 +49,4 @@ class Favicon internal constructor(
"mask-icon" -> -1
else -> 0
}
}
}

View File

@@ -2,7 +2,7 @@ package org.koitharu.kotatsu.parsers.model
class Favicons internal constructor(
favicons: Collection<Favicon>,
val referer: String,
@JvmField val referer: String,
) : Collection<Favicon> {
private val icons = favicons.sortedDescending()
@@ -18,6 +18,11 @@ class Favicons internal constructor(
override fun iterator(): Iterator<Favicon> = icons.iterator()
operator fun minus(victim: Favicon): Favicons = Favicons(
favicons = icons.filterNot { it == victim },
referer = referer,
)
/**
* Finds a favicon whose size in pixels is greater than or equal to the specified size.
* If such icon is not available returns the largest icon
@@ -42,4 +47,4 @@ class Favicons internal constructor(
}
return result
}
}
}

View File

@@ -6,67 +6,67 @@ class Manga(
/**
* Unique identifier for manga
*/
val id: Long,
@JvmField val id: Long,
/**
* Manga title, human-readable
*/
val title: String,
@JvmField val title: String,
/**
* Alternative title (for example on other language), may be null
*/
val altTitle: String?,
@JvmField val altTitle: String?,
/**
* Relative url to manga (**without** a domain) or any other uri.
* Used principally in parsers
*/
val url: String,
@JvmField val url: String,
/**
* Absolute url to manga, must be ready to open in browser
*/
val publicUrl: String,
@JvmField val publicUrl: String,
/**
* Normalized manga rating, must be in range of 0..1 or [RATING_UNKNOWN] if rating s unknown
* @see hasRating
*/
val rating: Float,
@JvmField val rating: Float,
/**
* Indicates that manga may contain sensitive information (18+, NSFW)
*/
val isNsfw: Boolean,
@JvmField val isNsfw: Boolean,
/**
* Absolute link to the cover
* @see largeCoverUrl
*/
val coverUrl: String,
@JvmField val coverUrl: String,
/**
* Tags (genres) of the manga
*/
val tags: Set<MangaTag>,
@JvmField val tags: Set<MangaTag>,
/**
* Manga status (ongoing, finished) or null if unknown
*/
val state: MangaState?,
@JvmField val state: MangaState?,
/**
* Author of the manga, may be null
*/
val author: String?,
@JvmField val author: String?,
/**
* Large cover url (absolute), null if is no large cover
* @see coverUrl
*/
val largeCoverUrl: String? = null,
@JvmField val largeCoverUrl: String? = null,
/**
* Manga description, may be html or null
*/
val description: String? = null,
@JvmField val description: String? = null,
/**
* List of chapters
*/
val chapters: List<MangaChapter>? = null,
@JvmField val chapters: List<MangaChapter>? = null,
/**
* Manga source
*/
val source: MangaSource,
@JvmField val source: MangaSource,
) {
/**
@@ -109,7 +109,7 @@ class Manga(
largeCoverUrl = largeCoverUrl,
description = description,
chapters = chapters,
source = source,
source = source
)
override fun equals(other: Any?): Boolean {
@@ -155,4 +155,8 @@ class Manga(
result = 31 * result + source.hashCode()
return result
}
}
override fun toString(): String {
return "Manga($id - \"$title\" [$url] - $source)"
}
}

View File

@@ -4,39 +4,61 @@ class MangaChapter(
/**
* An unique id of chapter
*/
val id: Long,
@JvmField val id: Long,
/**
* User-readable name of chapter
*/
val name: String,
@JvmField val name: String,
/**
* Chapter number starting from 1
* Chapter number starting from 1, 0 if unknown
*/
val number: Int,
@JvmField val number: Float,
/**
* Volume number starting from 1, 0 if unknown
*/
@JvmField val volume: Int,
/**
* Relative url to chapter (**without** a domain) or any other uri.
* Used principally in parsers
*/
val url: String,
@JvmField val url: String,
/**
* User-readable name of scanlator (releaser) or null if unknown
*/
val scanlator: String?,
@JvmField val scanlator: String?,
/**
* Chapter upload date in milliseconds
*/
val uploadDate: Long,
@JvmField val uploadDate: Long,
/**
* User-readable name of branch.
* A branch is a group of chapters that overlap (e.g. different languages)
*/
val branch: String?,
val source: MangaSource,
) : Comparable<MangaChapter> {
@JvmField val branch: String?,
@JvmField val source: MangaSource,
) {
override fun compareTo(other: MangaChapter): Int {
return number.compareTo(other.number)
}
@Deprecated(message = "Consider using constructor with volume value")
constructor(
id: Long,
name: String,
number: Int,
url: String,
scanlator: String?,
uploadDate: Long,
branch: String?,
source: MangaSource,
) : this(
id = id,
name = name,
number = number.toFloat(),
volume = 0,
url = url,
scanlator = scanlator,
uploadDate = uploadDate,
branch = branch,
source = source,
)
override fun equals(other: Any?): Boolean {
if (this === other) return true
@@ -47,6 +69,7 @@ class MangaChapter(
if (id != other.id) return false
if (name != other.name) return false
if (number != other.number) return false
if (volume != other.volume) return false
if (url != other.url) return false
if (scanlator != other.scanlator) return false
if (uploadDate != other.uploadDate) return false
@@ -59,7 +82,8 @@ class MangaChapter(
override fun hashCode(): Int {
var result = id.hashCode()
result = 31 * result + name.hashCode()
result = 31 * result + number
result = 31 * result + number.hashCode()
result = 31 * result + volume
result = 31 * result + url.hashCode()
result = 31 * result + (scanlator?.hashCode() ?: 0)
result = 31 * result + uploadDate.hashCode()
@@ -67,4 +91,20 @@ class MangaChapter(
result = 31 * result + source.hashCode()
return result
}
}
override fun toString(): String {
return "MangaChapter($id - #$number [$url] - $source)"
}
internal fun copy(volume: Int, number: Float) = MangaChapter(
id = id,
name = name,
number = number,
volume = volume,
url = url,
scanlator = scanlator,
uploadDate = uploadDate,
branch = branch,
source = source,
)
}

View File

@@ -0,0 +1,93 @@
package org.koitharu.kotatsu.parsers.model
import org.koitharu.kotatsu.parsers.MangaParser
import java.util.*
sealed interface MangaListFilter {
fun isEmpty(): Boolean
val sortOrder: SortOrder?
fun isValid(parser: MangaParser): Boolean = when (this) {
is Advanced -> (sortOrder in parser.availableSortOrders) &&
(tags.size <= 1 || parser.isMultipleTagsSupported) &&
(tagsExclude.isEmpty() || parser.isTagsExclusionSupported) &&
(contentRating.isEmpty() || parser.availableContentRating.containsAll(contentRating)) &&
(states.isEmpty() || parser.availableStates.containsAll(states))
is Search -> parser.isSearchSupported
}
data class Search(
@JvmField val query: String,
) : MangaListFilter {
override val sortOrder: SortOrder? = null
override fun isEmpty() = query.isBlank()
}
data class Advanced(
override val sortOrder: SortOrder,
@JvmField val tags: Set<MangaTag>,
@JvmField val tagsExclude: Set<MangaTag>,
@JvmField val locale: Locale?,
@JvmField val states: Set<MangaState>,
@JvmField val contentRating: Set<ContentRating>,
) : MangaListFilter {
override fun isEmpty(): Boolean =
tags.isEmpty() && tagsExclude.isEmpty() && locale == null && states.isEmpty() && contentRating.isEmpty()
fun newBuilder() = Builder(sortOrder)
.tags(tags)
.tagsExclude(tagsExclude)
.locale(locale)
.states(states)
.contentRatings(contentRating)
class Builder(sortOrder: SortOrder) {
private var _sortOrder: SortOrder = sortOrder
private var _tags: Set<MangaTag>? = null
private var _tagsExclude: Set<MangaTag>? = null
private var _locale: Locale? = null
private var _states: Set<MangaState>? = null
private var _contentRating: Set<ContentRating>? = null
fun sortOrder(order: SortOrder) = apply {
_sortOrder = order
}
fun tags(tags: Set<MangaTag>?) = apply {
_tags = tags
}
fun tagsExclude(tags: Set<MangaTag>?) = apply {
_tagsExclude = tags
}
fun locale(locale: Locale?) = apply {
_locale = locale
}
fun states(states: Set<MangaState>?) = apply {
_states = states
}
fun contentRatings(rating: Set<ContentRating>?) = apply {
_contentRating = rating
}
fun build() = Advanced(
sortOrder = _sortOrder,
tags = _tags.orEmpty(),
tagsExclude = _tagsExclude.orEmpty(),
locale = _locale,
states = _states.orEmpty(),
contentRating = _contentRating.orEmpty(),
)
}
}
}

View File

@@ -6,24 +6,19 @@ class MangaPage(
/**
* Unique identifier for manga
*/
val id: Long,
@JvmField val id: Long,
/**
* Relative url to page (**without** a domain) or any other uri.
* Used principally in parsers.
* May contain link to image or html page.
* @see MangaParser.getPageUrl
*/
val url: String,
/**
* Absolute link to the chapter or website home page.
* Used in Referer header
*/
val referer: String,
@JvmField val url: String,
/**
* Absolute url of the small page image if exists, null otherwise
*/
val preview: String?,
val source: MangaSource,
@JvmField val preview: String?,
@JvmField val source: MangaSource,
) {
override fun equals(other: Any?): Boolean {
@@ -34,19 +29,19 @@ class MangaPage(
if (id != other.id) return false
if (url != other.url) return false
if (referer != other.referer) return false
if (preview != other.preview) return false
if (source != other.source) return false
return true
return source == other.source
}
override fun hashCode(): Int {
var result = id.hashCode()
result = 31 * result + url.hashCode()
result = 31 * result + referer.hashCode()
result = 31 * result + (preview?.hashCode() ?: 0)
result = 31 * result + source.hashCode()
return result
}
}
override fun toString(): String {
return "MangaPage($id [$url] - $source)"
}
}

View File

@@ -1,5 +1,5 @@
package org.koitharu.kotatsu.parsers.model
enum class MangaState {
ONGOING, FINISHED
}
ONGOING, FINISHED, ABANDONED, PAUSED, UPCOMING
}

View File

@@ -6,13 +6,13 @@ class MangaTag(
/**
* User-readable tag title, should be in Title case
*/
val title: String,
@JvmField val title: String,
/**
* Identifier of a tag, must be unique among the source.
* @see MangaParser.getList
*/
val key: String,
val source: MangaSource,
@JvmField val key: String,
@JvmField val source: MangaSource,
) {
override fun equals(other: Any?): Boolean {
@@ -34,4 +34,8 @@ class MangaTag(
result = 31 * result + source.hashCode()
return result
}
}
override fun toString(): String {
return "MangaTag($key \"$title\" - $source)"
}
}

View File

@@ -5,5 +5,6 @@ enum class SortOrder {
POPULARITY,
RATING,
NEWEST,
ALPHABETICAL
}
ALPHABETICAL,
ALPHABETICAL_DESC
}

View File

@@ -0,0 +1,133 @@
package org.koitharu.kotatsu.parsers.network
import okhttp3.*
import okhttp3.MediaType.Companion.toMediaType
import okhttp3.RequestBody.Companion.toRequestBody
import org.json.JSONObject
import org.jsoup.HttpStatusException
import org.koitharu.kotatsu.parsers.exception.AuthRequiredException
import org.koitharu.kotatsu.parsers.exception.GraphQLException
import org.koitharu.kotatsu.parsers.exception.NotFoundException
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.util.await
import org.koitharu.kotatsu.parsers.util.parseJson
import java.net.HttpURLConnection
class OkHttpWebClient(
private val httpClient: OkHttpClient,
private val mangaSource: MangaSource,
) : WebClient {
override suspend fun httpGet(url: HttpUrl, extraHeaders: Headers?): Response {
val request = Request.Builder()
.get()
.url(url)
.addTags()
.addExtraHeaders(extraHeaders)
return httpClient.newCall(request.build()).await().ensureSuccess()
}
override suspend fun httpHead(url: HttpUrl): Response {
val request = Request.Builder()
.head()
.url(url)
.addTags()
return httpClient.newCall(request.build()).await().ensureSuccess()
}
override suspend fun httpPost(url: HttpUrl, form: Map<String, String>, extraHeaders: Headers?): Response {
val body = FormBody.Builder()
form.forEach { (k, v) ->
body.addEncoded(k, v)
}
val request = Request.Builder()
.post(body.build())
.url(url)
.addTags()
.addExtraHeaders(extraHeaders)
return httpClient.newCall(request.build()).await().ensureSuccess()
}
override suspend fun httpPost(url: HttpUrl, payload: String, extraHeaders: Headers?): Response {
val body = FormBody.Builder()
payload.split('&').forEach {
val pos = it.indexOf('=')
if (pos != -1) {
val k = it.substring(0, pos)
val v = it.substring(pos + 1)
body.addEncoded(k, v)
}
}
val request = Request.Builder()
.post(body.build())
.url(url)
.addTags()
.addExtraHeaders(extraHeaders)
return httpClient.newCall(request.build()).await().ensureSuccess()
}
override suspend fun httpPost(url: HttpUrl, body: JSONObject, extraHeaders: Headers?): Response {
val mediaType = "application/json; charset=utf-8".toMediaType()
val requestBody = body.toString().toRequestBody(mediaType)
val request = Request.Builder()
.post(requestBody)
.url(url)
.addTags()
.addExtraHeaders(extraHeaders)
return httpClient.newCall(request.build()).await().ensureSuccess()
}
override suspend fun graphQLQuery(endpoint: String, query: String): JSONObject {
val body = JSONObject()
body.put("operationName", null as Any?)
body.put("variables", JSONObject())
body.put("query", "{$query}")
val mediaType = "application/json; charset=utf-8".toMediaType()
val requestBody = body.toString().toRequestBody(mediaType)
val request = Request.Builder()
.post(requestBody)
.url(endpoint)
.addTags()
val json = httpClient.newCall(request.build()).await().parseJson()
json.optJSONArray("errors")?.let {
if (it.length() != 0) {
throw GraphQLException(it)
}
}
return json
}
private fun Request.Builder.addTags(): Request.Builder {
tag(MangaSource::class.java, mangaSource)
return this
}
private fun Request.Builder.addExtraHeaders(headers: Headers?): Request.Builder {
if (headers != null) {
headers(headers)
}
return this
}
private fun Response.ensureSuccess(): Response {
val exception: Exception? = when (code) { // Catch some error codes, not all
HttpURLConnection.HTTP_NOT_FOUND -> NotFoundException(message, request.url.toString())
HttpURLConnection.HTTP_UNAUTHORIZED -> request.tag(MangaSource::class.java)?.let {
AuthRequiredException(it)
} ?: HttpStatusException(message, code, request.url.toString())
in 400..599 -> HttpStatusException(message, code, request.url.toString())
else -> null
}
if (exception != null) {
runCatching {
close()
}.onFailure {
exception.addSuppressed(it)
}
throw exception
}
return this
}
}

View File

@@ -0,0 +1,14 @@
package org.koitharu.kotatsu.parsers.network
object UserAgents {
const val CHROME_MOBILE =
"Mozilla/5.0 (Linux; Android 13) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.5735.196 Mobile Safari/537.36"
const val CHROME_DESKTOP =
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36"
const val FIREFOX_DESKTOP = "Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/116.0"
const val KOTATSU = "Kotatsu/5.3 (Android 13;;; en)"
}

View File

@@ -0,0 +1,116 @@
package org.koitharu.kotatsu.parsers.network
import okhttp3.Headers
import okhttp3.HttpUrl
import okhttp3.HttpUrl.Companion.toHttpUrl
import okhttp3.Response
import org.json.JSONObject
interface WebClient {
/**
* Do a GET http request to specific url
* @param url
*/
suspend fun httpGet(url: String): Response = httpGet(url.toHttpUrl())
suspend fun httpGet(url: String, extraHeaders: Headers?): Response = httpGet(url.toHttpUrl(), extraHeaders)
/**
* Do a GET http request to specific url
* @param url
*/
suspend fun httpGet(url: HttpUrl): Response = httpGet(url, null)
/**
* Do a GET http request to specific url
* @param url
* @param extraHeaders additional HTTP headers for request
*/
suspend fun httpGet(url: HttpUrl, extraHeaders: Headers?): Response
/**
* Do a HEAD http request to specific url
* @param url
*/
suspend fun httpHead(url: String): Response = httpHead(url.toHttpUrl())
/**
* Do a HEAD http request to specific url
* @param url
*/
suspend fun httpHead(url: HttpUrl): Response
/**
* Do a POST http request to specific url with `multipart/form-data` payload
* @param url
* @param form payload as key=>value map
*/
suspend fun httpPost(url: String, form: Map<String, String>): Response = httpPost(url.toHttpUrl(), form, null)
/**
* Do a POST http request to specific url with `multipart/form-data` payload
* @param url
* @param form payload as key=>value map
*/
suspend fun httpPost(url: HttpUrl, form: Map<String, String>): Response = httpPost(url, form, null)
/**
* Do a POST http request to specific url with `multipart/form-data` payload
* @param url
* @param form payload as key=>value map
* @param extraHeaders additional HTTP headers for request
*/
suspend fun httpPost(url: HttpUrl, form: Map<String, String>, extraHeaders: Headers?): Response
/**
* Do a POST http request to specific url with `multipart/form-data` payload
* @param url
* @param payload payload as `key=value` string with `&` separator
*/
suspend fun httpPost(url: String, payload: String): Response = httpPost(url.toHttpUrl(), payload, null)
/**
* Do a POST http request to specific url with `multipart/form-data` payload
* @param url
* @param payload payload as `key=value` string with `&` separator
*/
suspend fun httpPost(url: HttpUrl, payload: String): Response = httpPost(url, payload, null)
/**
* Do a POST http request to specific url with `multipart/form-data` payload
* @param url
* @param payload payload as `key=value` string with `&` separator
* @param extraHeaders additional HTTP headers for request
*/
suspend fun httpPost(url: HttpUrl, payload: String, extraHeaders: Headers?): Response
/**
* Do a POST http request to specific url with json payload
* @param url
* @param body
*/
suspend fun httpPost(url: String, body: JSONObject): Response = httpPost(url.toHttpUrl(), body, null)
/**
* Do a POST http request to specific url with json payload
* @param url
* @param body
*/
suspend fun httpPost(url: HttpUrl, body: JSONObject): Response = httpPost(url, body, null)
/**
* Do a POST http request to specific url with json payload
* @param url
* @param body
* @param extraHeaders additional HTTP headers for request
*/
suspend fun httpPost(url: HttpUrl, body: JSONObject, extraHeaders: Headers?): Response
/**
* Do a GraphQL request to specific url
* @param endpoint an url
* @param query GraphQL request payload
*/
suspend fun graphQLQuery(endpoint: String, query: String): JSONObject
}

View File

@@ -1,216 +0,0 @@
package org.koitharu.kotatsu.parsers.site
import androidx.collection.ArraySet
import androidx.collection.SparseArrayCompat
import org.json.JSONArray
import org.json.JSONObject
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaParser
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import org.koitharu.kotatsu.parsers.util.json.*
import java.text.SimpleDateFormat
import java.util.*
/**
* https://api.comick.fun/docs/static/index.html
*/
private const val PAGE_SIZE = 20
private const val CHAPTERS_LIMIT = 99999
@MangaSourceParser("COMICK_FUN", "ComicK")
internal class ComickFunParser(override val context: MangaLoaderContext) : MangaParser(MangaSource.COMICK_FUN) {
override val configKeyDomain = ConfigKey.Domain("comick.fun", null)
override val sortOrders: Set<SortOrder> = EnumSet.of(
SortOrder.POPULARITY,
SortOrder.UPDATED,
SortOrder.RATING,
)
@Volatile
private var cachedTags: SparseArrayCompat<MangaTag>? = null
override suspend fun getList(
offset: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val domain = getDomain()
val url = buildString {
append("https://api.")
append(domain)
append("/search?tachiyomi=true")
if (!query.isNullOrEmpty()) {
if (offset > 0) {
return emptyList()
}
append("&q=")
append(query.urlEncoded())
} else {
append("&limit=")
append(PAGE_SIZE)
append("&page=")
append((offset / PAGE_SIZE) + 1)
if (!tags.isNullOrEmpty()) {
append("&genres=")
appendAll(tags, "&genres=", MangaTag::key)
}
append("&sort=") // view, uploaded, rating, follow, user_follow_count
append(
when (sortOrder) {
SortOrder.POPULARITY -> "view"
SortOrder.RATING -> "rating"
else -> "uploaded"
},
)
}
}
val ja = context.httpGet(url).parseJsonArray()
val tagsMap = cachedTags ?: loadTags()
return ja.mapJSON { jo ->
val slug = jo.getString("slug")
Manga(
id = generateUid(slug),
title = jo.getString("title"),
altTitle = null,
url = slug,
publicUrl = "https://$domain/comic/$slug",
rating = jo.getDoubleOrDefault("rating", -10.0).toFloat() / 10f,
isNsfw = false,
coverUrl = jo.getString("cover_url"),
largeCoverUrl = null,
description = jo.getStringOrNull("desc"),
tags = jo.selectGenres("genres", tagsMap),
state = runCatching {
if (jo.getBoolean("translation_completed")) {
MangaState.FINISHED
} else {
MangaState.ONGOING
}
}.getOrNull(),
author = null,
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga {
val domain = getDomain()
val url = "https://api.$domain/comic/${manga.url}?tachiyomi=true"
val jo = context.httpGet(url).parseJson()
val comic = jo.getJSONObject("comic")
return manga.copy(
title = comic.getString("title"),
altTitle = null, // TODO
isNsfw = jo.getBoolean("matureContent") || comic.getBoolean("hentai"),
description = comic.getStringOrNull("parsed") ?: comic.getString("desc"),
tags = manga.tags + jo.getJSONArray("genres").mapJSONToSet {
MangaTag(
title = it.getString("name"),
key = it.getString("slug"),
source = source,
)
},
author = jo.getJSONArray("artists").optJSONObject(0)?.getString("name"),
chapters = getChapters(comic.getLong("id")),
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val jo = context.httpGet(
"https://api.${getDomain()}/chapter/${chapter.url}?tachiyomi=true",
).parseJson().getJSONObject("chapter")
val referer = "https://${getDomain()}/"
return jo.getJSONArray("images").mapJSON {
val url = it.getString("url")
MangaPage(
id = generateUid(url),
url = url,
referer = referer,
preview = null,
source = source,
)
}
}
override suspend fun getTags(): Set<MangaTag> {
val sparseArray = cachedTags ?: loadTags()
val set = ArraySet<MangaTag>(sparseArray.size())
for (i in 0 until sparseArray.size()) {
set.add(sparseArray.valueAt(i))
}
return set
}
private suspend fun loadTags(): SparseArrayCompat<MangaTag> {
val ja = context.httpGet("https://api.${getDomain()}/genre").parseJsonArray()
val tags = SparseArrayCompat<MangaTag>(ja.length())
for (jo in ja.JSONIterator()) {
tags.append(
jo.getInt("id"),
MangaTag(
title = jo.getString("name"),
key = jo.getString("slug"),
source = source,
),
)
}
cachedTags = tags
return tags
}
private suspend fun getChapters(id: Long): List<MangaChapter> {
val ja = context.httpGet(
url = "https://api.${getDomain()}/comic/$id/chapter?tachiyomi=true&limit=$CHAPTERS_LIMIT",
).parseJson().getJSONArray("chapters")
val dateFormat = SimpleDateFormat("yyyy-MM-dd")
val counters = HashMap<Locale, Int>()
return ja.mapReversed { jo ->
val locale = Locale.forLanguageTag(jo.getString("lang"))
var number = counters[locale] ?: 0
number++
counters[locale] = number
MangaChapter(
id = generateUid(jo.getLong("id")),
name = buildString {
jo.getStringOrNull("vol")?.let { append("Vol ").append(it).append(' ') }
jo.getStringOrNull("chap")?.let { append("Chap ").append(it) }
jo.getStringOrNull("title")?.let { append(": ").append(it) }
},
number = number,
url = jo.getString("hid"),
scanlator = jo.optJSONArray("group_name")?.optString(0),
uploadDate = dateFormat.tryParse(jo.getString("created_at").substringBefore('T')),
branch = locale.getDisplayName(locale).toTitleCase(locale),
source = source,
)
}
}
private inline fun <R> JSONArray.mapReversed(block: (JSONObject) -> R): List<R> {
val len = length()
val destination = ArrayList<R>(len)
for (i in (0 until len).reversed()) {
val jo = getJSONObject(i)
destination.add(block(jo))
}
return destination
}
private fun JSONObject.selectGenres(name: String, tags: SparseArrayCompat<MangaTag>): Set<MangaTag> {
val array = optJSONArray(name) ?: return emptySet()
val res = ArraySet<MangaTag>(array.length())
for (i in 0 until array.length()) {
val id = array.getInt(i)
val tag = tags.get(id) ?: continue
res.add(tag)
}
return res
}
}

View File

@@ -1,279 +0,0 @@
package org.koitharu.kotatsu.parsers.site
import org.jsoup.nodes.Element
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaParserAuthProvider
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.exception.AuthRequiredException
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.util.*
import kotlin.math.pow
private const val DOMAIN_UNAUTHORIZED = "e-hentai.org"
private const val DOMAIN_AUTHORIZED = "exhentai.org"
@MangaSourceParser("EXHENTAI", "ExHentai")
internal class ExHentaiParser(
override val context: MangaLoaderContext,
) : PagedMangaParser(MangaSource.EXHENTAI, pageSize = 25), MangaParserAuthProvider {
override val sortOrders: Set<SortOrder> = Collections.singleton(
SortOrder.NEWEST,
)
override val configKeyDomain: ConfigKey.Domain
get() = ConfigKey.Domain(if (isAuthorized) DOMAIN_AUTHORIZED else DOMAIN_UNAUTHORIZED, null)
override val authUrl: String
get() = "https://${getDomain()}/bounce_login.php"
private val ratingPattern = Regex("-?[0-9]+px")
private val authCookies = arrayOf("ipb_member_id", "ipb_pass_hash")
private var updateDm = false
override val isAuthorized: Boolean
get() {
val authorized = isAuthorized(DOMAIN_UNAUTHORIZED)
if (authorized) {
if (!isAuthorized(DOMAIN_AUTHORIZED)) {
context.cookieJar.copyCookies(
DOMAIN_UNAUTHORIZED,
DOMAIN_AUTHORIZED,
authCookies,
)
context.cookieJar.insertCookies(DOMAIN_AUTHORIZED, "yay=louder")
}
return true
}
return false
}
init {
context.cookieJar.insertCookies(DOMAIN_AUTHORIZED, "nw=1", "sl=dm_2")
context.cookieJar.insertCookies(DOMAIN_UNAUTHORIZED, "nw=1", "sl=dm_2")
}
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
var search = query?.urlEncoded().orEmpty()
val url = buildString {
append("https://")
append(getDomain())
append("/?page=")
append(page)
if (!tags.isNullOrEmpty()) {
var fCats = 0
for (tag in tags) {
tag.key.toIntOrNull()?.let { fCats = fCats or it } ?: run {
search += tag.key + " "
}
}
if (fCats != 0) {
append("&f_cats=")
append(1023 - fCats)
}
}
if (search.isNotEmpty()) {
append("&f_search=")
append(search.trim().replace(' ', '+'))
}
// by unknown reason cookie "sl=dm_2" is ignored, so, we should request it again
if (updateDm) {
append("&inline_set=dm_e")
}
}
val body = context.httpGet(url).parseHtml().body()
val root = body.selectFirst("table.itg")
?.selectFirst("tbody")
?: if (updateDm) {
body.parseFailed("Cannot find root")
} else {
updateDm = true
return getListPage(page, query, tags, sortOrder)
}
updateDm = false
return root.children().mapNotNull { tr ->
if (tr.childrenSize() != 2) return@mapNotNull null
val (td1, td2) = tr.children()
val glink = td2.selectFirstOrThrow("div.glink")
val a = glink.parents().select("a").first() ?: glink.parseFailed("link not found")
val href = a.attrAsRelativeUrl("href")
val tagsDiv = glink.nextElementSibling() ?: glink.parseFailed("tags div not found")
val mainTag = td2.selectFirst("div.cn")?.let { div ->
MangaTag(
title = div.text().toTitleCase(),
key = tagIdByClass(div.classNames()) ?: return@let null,
source = source,
)
}
Manga(
id = generateUid(href),
title = glink.text().cleanupTitle(),
altTitle = null,
url = href,
publicUrl = a.absUrl("href"),
rating = td2.selectFirst("div.ir")?.parseRating() ?: RATING_UNKNOWN,
isNsfw = true,
coverUrl = td1.selectFirst("img")?.absUrl("src").orEmpty(),
tags = setOfNotNull(mainTag),
state = null,
author = tagsDiv.getElementsContainingOwnText("artist:").first()
?.nextElementSibling()?.text(),
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga {
val doc = context.httpGet(manga.url.toAbsoluteUrl(getDomain())).parseHtml()
val root = doc.body().selectFirstOrThrow("div.gm")
val cover = root.getElementById("gd1")?.children()?.first()
val title = root.getElementById("gd2")
val taglist = root.getElementById("taglist")
val tabs = doc.body().selectFirst("table.ptt")?.selectFirst("tr")
return manga.copy(
title = title?.getElementById("gn")?.text()?.cleanupTitle() ?: manga.title,
altTitle = title?.getElementById("gj")?.text()?.cleanupTitle() ?: manga.altTitle,
publicUrl = doc.baseUri().ifEmpty { manga.publicUrl },
rating = root.getElementById("rating_label")?.text()
?.substringAfterLast(' ')
?.toFloatOrNull()
?.div(5f) ?: manga.rating,
largeCoverUrl = cover?.styleValueOrNull("background")?.cssUrl(),
description = taglist?.select("tr")?.joinToString("<br>") { tr ->
val (tc, td) = tr.children()
val subtags = td.select("a").joinToString { it.html() }
"<b>${tc.html()}</b> $subtags"
},
chapters = tabs?.select("a")?.findLast { a ->
a.text().toIntOrNull() != null
}?.let { a ->
val count = a.text().toInt()
val chapters = ChaptersListBuilder(count)
for (i in 1..count) {
val url = "${manga.url}?p=${i - 1}"
chapters += MangaChapter(
id = generateUid(url),
name = "${manga.title} #$i",
number = i,
url = url,
uploadDate = 0L,
source = source,
scanlator = null,
branch = null,
)
}
chapters.toList()
},
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val doc = context.httpGet(chapter.url.toAbsoluteUrl(getDomain())).parseHtml()
val root = doc.body().requireElementById("gdt")
return root.select("a").map { a ->
val url = a.attrAsRelativeUrl("href")
MangaPage(
id = generateUid(url),
url = url,
referer = a.absUrl("href"),
preview = null,
source = source,
)
}
}
override suspend fun getPageUrl(page: MangaPage): String {
val doc = context.httpGet(page.url.toAbsoluteUrl(getDomain())).parseHtml()
return doc.body().requireElementById("img").attrAsAbsoluteUrl("src")
}
override suspend fun getTags(): Set<MangaTag> {
val doc = context.httpGet("https://${getDomain()}").parseHtml()
val root = doc.body().requireElementById("searchbox").selectFirstOrThrow("table")
return root.select("div.cs").mapNotNullToSet { div ->
val id = div.id().substringAfterLast('_').toIntOrNull()
?: return@mapNotNullToSet null
MangaTag(
title = div.text().toTitleCase(),
key = id.toString(),
source = source,
)
}
}
override suspend fun getUsername(): String {
val doc = context.httpGet("https://forums.$DOMAIN_UNAUTHORIZED/").parseHtml().body()
val username = doc.getElementById("userlinks")
?.getElementsByAttributeValueContaining("href", "showuser=")
?.firstOrNull()
?.ownText()
?: if (doc.getElementById("userlinksguest") != null) {
throw AuthRequiredException(source)
} else {
doc.parseFailed()
}
return username
}
private fun isAuthorized(domain: String): Boolean {
val cookies = context.cookieJar.getCookies(domain).mapToSet { x -> x.name }
return authCookies.all { it in cookies }
}
private fun Element.parseRating(): Float {
return runCatching {
val style = requireNotNull(attr("style"))
val (v1, v2) = ratingPattern.find(style)!!.destructured
var p1 = v1.dropLast(2).toInt()
val p2 = v2.dropLast(2).toInt()
if (p2 != -1) {
p1 += 8
}
(80 - p1) / 80f
}.getOrDefault(RATING_UNKNOWN)
}
private fun String.cleanupTitle(): String {
val result = StringBuilder(length)
var skip = false
for (c in this) {
when {
c == '[' -> skip = true
c == ']' -> skip = false
c.isWhitespace() && result.isEmpty() -> continue
!skip -> result.append(c)
}
}
while (result.lastOrNull()?.isWhitespace() == true) {
result.deleteCharAt(result.lastIndex)
}
return result.toString()
}
private fun String.cssUrl(): String? {
val fromIndex = indexOf("url(")
if (fromIndex == -1) {
return null
}
val toIndex = indexOf(')', startIndex = fromIndex)
return if (toIndex == -1) {
null
} else {
substring(fromIndex + 4, toIndex).trim()
}
}
private fun tagIdByClass(classNames: Collection<String>): String? {
val className = classNames.find { x -> x.startsWith("ct") } ?: return null
val num = className.drop(2).toIntOrNull(16) ?: return null
return 2.0.pow(num).toInt().toString()
}
}

View File

@@ -1,186 +0,0 @@
package org.koitharu.kotatsu.parsers.site
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaParser
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("MANGAOWL", "MangaOwl", "en")
internal class MangaOwlParser(override val context: MangaLoaderContext) : MangaParser(MangaSource.MANGAOWL) {
override val configKeyDomain = ConfigKey.Domain("mangaowls.com", null)
override val sortOrders: Set<SortOrder> = EnumSet.of(
SortOrder.POPULARITY,
SortOrder.NEWEST,
SortOrder.UPDATED,
)
private val regexNsfw = Regex("(yaoi)|(yuri)|(smut)|(mature)|(adult)", RegexOption.IGNORE_CASE)
override suspend fun getList(
offset: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val page = (offset / 36f).toIntUp().inc()
val link = buildString {
append("https://")
append(getDomain())
when {
!query.isNullOrEmpty() -> {
append("/search/$page?search=")
append(query.urlEncoded())
}
!tags.isNullOrEmpty() -> {
for (tag in tags) {
append(tag.key)
}
append("/$page?type=${getAlternativeSortKey(sortOrder)}")
}
else -> {
append("/${getSortKey(sortOrder)}/$page")
}
}
}
val doc = context.httpGet(link).parseHtml()
val slides = doc.body().selectOrThrow("ul.slides")
val items = slides.select("div.col-md-2")
return items.mapNotNull { item ->
val href = item.selectFirst("h6 a")?.attrAsRelativeUrlOrNull("href") ?: return@mapNotNull null
Manga(
id = generateUid(href),
title = item.selectFirst("h6 a")?.text() ?: return@mapNotNull null,
coverUrl = item.select("div.img-responsive").attr("abs:data-background-image"),
altTitle = null,
author = null,
rating = runCatching {
item.selectFirst("div.block-stars")
?.text()
?.toFloatOrNull()
?.div(10f)
}.getOrNull() ?: RATING_UNKNOWN,
url = href,
isNsfw = false,
tags = emptySet(),
state = null,
publicUrl = href.toAbsoluteUrl(getDomain()),
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga {
val doc = context.httpGet(manga.publicUrl).parseHtml()
val info = doc.body().selectFirstOrThrow("div.single_detail")
val table = doc.body().selectFirstOrThrow("div.single-grid-right")
val dateFormat = SimpleDateFormat("MM/dd/yyyy", Locale.US)
val trRegex = "window\\['tr'] = '([^']*)';".toRegex(RegexOption.IGNORE_CASE)
val trElement = doc.getElementsByTag("script").find { trRegex.find(it.data()) != null }
?: doc.parseFailed("Oops, tr not found")
val tr = trRegex.find(trElement.data())!!.groups[1]!!.value
val s = context.encodeBase64(getDomain().toByteArray())
var isNsfw = manga.isNsfw
val parsedTags = info.select("div.col-xs-12.col-md-8.single-right-grid-right > p > a[href*=genres]")
.mapNotNullToSet {
val a = it.selectFirst("a") ?: return@mapNotNullToSet null
val name = a.text()
if (!isNsfw && isNsfwGenre(name)) {
isNsfw = true
}
MangaTag(
title = name.toTitleCase(),
key = a.attr("href"),
source = source,
)
}
return manga.copy(
description = info.selectFirst(".description")?.html(),
largeCoverUrl = info.select("img").first()?.let { img ->
if (img.hasAttr("data-src")) img.attr("abs:data-src") else img.attr("abs:src")
},
isNsfw = isNsfw,
author = info.selectFirst("p.fexi_header_para a.author_link")?.text(),
state = parseStatus(info.select("p.fexi_header_para:contains(status)").first()?.ownText()),
tags = manga.tags + parsedTags,
chapters = table.select("div.table.table-chapter-list").select("li.list-group-item.chapter_list")
.asReversed().mapChapters { i, li ->
val a = li.select("a")
val href = a.attr("data-href").ifEmpty {
li.parseFailed("Link is missing")
}
MangaChapter(
id = generateUid(href),
name = a.select("label").text(),
number = i + 1,
url = "$href?tr=$tr&s=$s",
scanlator = null,
branch = null,
uploadDate = dateFormat.tryParse(li.selectFirst("small:last-of-type")?.text()),
source = MangaSource.MANGAOWL,
)
},
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(getDomain())
val doc = context.httpGet(fullUrl).parseHtml()
val root = doc.body().selectOrThrow("div.item img.owl-lazy")
return root.map { div ->
val url = div?.attrAsRelativeUrlOrNull("data-src") ?: doc.parseFailed("Page image not found")
MangaPage(
id = generateUid(url),
url = url,
preview = null,
referer = url,
source = MangaSource.MANGAOWL,
)
}
}
private fun parseStatus(status: String?) = when {
status == null -> null
status.contains("Ongoing") -> MangaState.ONGOING
status.contains("Completed") -> MangaState.FINISHED
else -> null
}
override suspend fun getTags(): Set<MangaTag> {
val doc = context.httpGet("https://${getDomain()}/").parseHtml()
val root = doc.body().select("ul.dropdown-menu.multi-column.columns-3").select("li")
return root.mapToSet { p ->
val a = p.selectFirstOrThrow("a")
MangaTag(
title = a.text().toTitleCase(),
key = a.attr("href"),
source = source,
)
}
}
private fun getSortKey(sortOrder: SortOrder) =
when (sortOrder) {
SortOrder.POPULARITY -> "popular"
SortOrder.NEWEST -> "new_release"
SortOrder.UPDATED -> "lastest"
else -> "lastest"
}
private fun getAlternativeSortKey(sortOrder: SortOrder) =
when (sortOrder) {
SortOrder.POPULARITY -> "0"
SortOrder.NEWEST -> "2"
SortOrder.UPDATED -> "3"
else -> "3"
}
private fun isNsfwGenre(name: String): Boolean = regexNsfw.containsMatchIn(name)
}

View File

@@ -1,188 +0,0 @@
package org.koitharu.kotatsu.parsers.site
import androidx.collection.ArraySet
import kotlinx.coroutines.async
import kotlinx.coroutines.awaitAll
import kotlinx.coroutines.coroutineScope
import org.jsoup.nodes.Element
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("NHENTAI", "N-Hentai")
class NHentaiParser(override val context: MangaLoaderContext) : PagedMangaParser(MangaSource.NHENTAI, pageSize = 25) {
override val configKeyDomain: ConfigKey.Domain
get() = ConfigKey.Domain("nhentai.net", null)
override val sortOrders: Set<SortOrder>
get() = EnumSet.of(SortOrder.NEWEST, SortOrder.POPULARITY)
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
if (query.isNullOrEmpty() && tags != null && tags.size > 1) {
return getListPage(page, buildQuery(tags), emptySet(), sortOrder)
}
val domain = getDomain()
val url = buildString {
append("https://")
append(domain)
if (!query.isNullOrEmpty()) {
append("/search/?q=")
append(query.urlEncoded())
append("&page=")
append(page)
if (sortOrder == SortOrder.POPULARITY) {
append("&sort=popular")
}
} else {
append('/')
if (!tags.isNullOrEmpty()) {
val tag = tags.single()
append("tag/")
append(tag.key)
append('/')
if (sortOrder == SortOrder.POPULARITY) {
append("popular")
}
append("?page=")
append(page)
} else {
if (sortOrder == SortOrder.POPULARITY) {
append("?sort=popular&page=")
} else {
append("?page=")
}
append(page)
}
}
}
val root = context.httpGet(url).parseHtml().body().requireElementById("content")
.selectLastOrThrow("div.index-container")
val regexBrackets = Regex("\\[[^]]+]|\\([^)]+\\)")
val regexSpaces = Regex("\\s+")
return root.select(".gallery").map { div ->
val a = div.selectFirstOrThrow("a.cover")
val href = a.attrAsRelativeUrl("href")
val img = div.selectFirstOrThrow("img")
val title = div.selectFirstOrThrow(".caption").text()
Manga(
id = generateUid(href),
title = title.replace(regexBrackets, "")
.replace(regexSpaces, " ")
.trim(),
altTitle = null,
url = href,
publicUrl = href.toAbsoluteUrl(domain),
rating = RATING_UNKNOWN,
isNsfw = true,
coverUrl = img.attrAsAbsoluteUrlOrNull("data-src")
?: img.attrAsAbsoluteUrl("src"),
tags = setOf(),
state = null,
author = null,
largeCoverUrl = null,
description = null,
chapters = listOf(),
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga {
val root = context.httpGet(
url = manga.url.toAbsoluteUrl(getDomain()),
).parseHtml().body().requireElementById("bigcontainer")
val img = root.requireElementById("cover").selectFirstOrThrow("img")
val tagContainers = root.requireElementById("tags").select(".tag-container")
val dateFormat = SimpleDateFormat(
"yyyy-MM-dd'T'HH:mm:ss.SSSSSS'+00:00'",
Locale.ROOT,
)
return manga.copy(
tags = tagContainers.find { x -> x.ownText() == "Tags:" }?.parseTags() ?: manga.tags,
author = tagContainers.find { x -> x.ownText() == "Artists:" }
?.selectFirst("span.name")?.text()?.toCamelCase(),
largeCoverUrl = img.attrAsAbsoluteUrlOrNull("data-src")
?: img.attrAsAbsoluteUrl("src"),
description = null,
chapters = listOf(
MangaChapter(
id = manga.id,
name = manga.title,
number = 1,
url = manga.url,
scanlator = null,
uploadDate = dateFormat.tryParse(
tagContainers.find { x -> x.ownText() == "Uploaded:" }
?.selectFirst("time")
?.attr("datetime"),
),
branch = null,
source = source,
),
),
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val url = chapter.url.toAbsoluteUrl(getDomain())
val root = context.httpGet(url).parseHtml().requireElementById("thumbnail-container")
return root.select(".thumb-container").map { div ->
val a = div.selectFirstOrThrow("a")
val img = div.selectFirstOrThrow("img")
val href = a.attrAsRelativeUrl("href")
MangaPage(
id = generateUid(href),
url = href,
referer = url,
preview = img.attrAsAbsoluteUrlOrNull("data-src")
?: img.attrAsAbsoluteUrl("src"),
source = source,
)
}
}
override suspend fun getPageUrl(page: MangaPage): String {
val root = context.httpGet(page.url.toAbsoluteUrl(getDomain())).parseHtml().body()
.requireElementById("image-container")
return root.selectFirstOrThrow("img").attrAsAbsoluteUrl("src")
}
override suspend fun getTags(): Set<MangaTag> {
return coroutineScope {
// parse first 3 pages of tags
(1..3).map { page ->
async { getTags(page) }
}
}.awaitAll().flattenTo(ArraySet(360))
}
private suspend fun getTags(page: Int): Set<MangaTag> {
val root = context.httpGet("https://${getDomain()}/tags/popular?page=$page").parseHtml().body()
.getElementById("tag-container")
return root?.parseTags().orEmpty()
}
private fun Element.parseTags() = select("a.tag").mapToSet { a ->
val href = a.attr("href").removeSuffix('/')
MangaTag(
title = a.selectFirstOrThrow(".name").text().toTitleCase(),
key = href.substringAfterLast('/'),
source = source,
)
}
private fun buildQuery(tags: Collection<MangaTag>) = tags.joinToString(separator = " ") { tag ->
"tag:\"${tag.key}\""
}
}

View File

@@ -1,221 +0,0 @@
package org.koitharu.kotatsu.parsers.site
import androidx.collection.ArrayMap
import androidx.collection.ArraySet
import kotlinx.coroutines.sync.Mutex
import kotlinx.coroutines.sync.withLock
import org.koitharu.kotatsu.parsers.*
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.exception.NotFoundException
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("NETTRUYEN", "NetTruyen", "vi")
class NetTruyenParser(override val context: MangaLoaderContext) :
PagedMangaParser(MangaSource.NETTRUYEN, pageSize = 36) {
override val configKeyDomain: ConfigKey.Domain
get() = ConfigKey.Domain("www.nettruyenme.com", null)
override val sortOrders: Set<SortOrder>
get() = EnumSet.of(SortOrder.UPDATED, SortOrder.POPULARITY, SortOrder.NEWEST, SortOrder.RATING)
private val mutex = Mutex()
private val dateFormat = SimpleDateFormat("dd/MM/yy", Locale.US)
private var tagCache: ArrayMap<String, MangaTag>? = null
override suspend fun getDetails(manga: Manga): Manga {
val doc = context.httpGet(manga.url.toAbsoluteUrl(getDomain())).parseHtml()
val rating = doc.selectFirst("span[itemprop=ratingValue]")
?.ownText()
?.toFloatOrNull() ?: 0f
val chapterElements = doc.getElementById("nt_listchapter")?.select("ul > li") ?: doc.parseFailed()
val chapters = chapterElements.asReversed().mapChapters { index, element ->
val a = element.selectFirst("div.chapter > a") ?: return@mapChapters null
val relativeUrl = a.attrAsRelativeUrlOrNull("href") ?: return@mapChapters null
val timeText = element.selectFirst("div.col-xs-4.text-center.no-wrap.small")?.text()
MangaChapter(
id = generateUid(relativeUrl),
name = a.text(),
number = index + 1,
url = relativeUrl,
scanlator = null,
uploadDate = parseChapterTime(timeText),
branch = null,
source = source,
)
}
return manga.copy(
rating = rating / 5,
chapters = chapters,
description = doc.selectFirst("div.detail-content > p")?.html(),
isNsfw = doc.selectFirst("div.alert.alert-danger > strong:contains(Cảnh báo độ tuổi)") != null,
)
}
// 20 giây trước
// 52 phút trước
// 6 giờ trước
// 2 ngày trước
// 19:09 30/07
// 23/12/21
private fun parseChapterTime(timeText: String?): Long {
if (timeText.isNullOrEmpty()) {
return 0L
}
val timeWords = arrayOf("giây", "phút", "giờ", "ngày")
val calendar = Calendar.getInstance()
val timeArr = timeText.split(' ')
if (WordSet(*timeWords).anyWordIn(timeText)) {
val timeSuffix = timeArr.getOrNull(1)
val timeDiff = timeArr.getOrNull(0)?.toIntOrNull() ?: return 0L
when (timeSuffix) {
timeWords[0] -> calendar.add(Calendar.SECOND, -timeDiff)
timeWords[1] -> calendar.add(Calendar.MINUTE, -timeDiff)
timeWords[2] -> calendar.add(Calendar.HOUR, -timeDiff)
timeWords[3] -> calendar.add(Calendar.DATE, -timeDiff)
else -> return 0L
}
} else {
val relativeDate = timeArr.lastOrNull() ?: return 0L
val dateString = when (relativeDate.split('/').size) {
2 -> {
val currentYear = calendar.get(Calendar.YEAR).toString().takeLast(2)
"$relativeDate/$currentYear"
}
3 -> relativeDate
else -> return 0L
}
calendar.timeInMillis = dateFormat.tryParse(dateString)
}
return calendar.time.time
}
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val isSearching = !query.isNullOrEmpty()
val url = buildString {
append("https://")
append(getDomain())
if (isSearching) {
append("/tim-truyen?keyword=")
append(query!!.urlEncoded())
append("&page=")
append(page)
} else {
val tagQuery = tags.orEmpty().joinToString(",") { it.key }
append("/tim-truyen-nang-cao?genres=$tagQuery")
append("&notgenres=&gender=-1&status=-1&minchapter=1&sort=${getSortOrderKey(sortOrder)}")
append("&page=$page")
}
}
val response = if (isSearching) {
val result = runCatching { context.httpGet(url) }
val exception = result.exceptionOrNull()
if (exception is NotFoundException) {
return emptyList()
}
result.getOrThrow()
} else {
context.httpGet(url)
}
val itemsElements = response.parseHtml()
.select("div.ModuleContent > div.items")
.select("div.item")
return itemsElements.mapNotNull { item ->
val tooltipElement = item.selectFirst("div.box_tootip") ?: return@mapNotNull null
val absUrl = item.selectFirst("div.image > a")?.attrAsAbsoluteUrlOrNull("href") ?: return@mapNotNull null
val slug = absUrl.substringAfterLast('/')
val mangaState = when (tooltipElement.selectFirst("div.message_main > p:contains(Tình trạng)")?.ownText()) {
"Đang tiến hành" -> MangaState.ONGOING
"Hoàn thành" -> MangaState.FINISHED
else -> null
}
val tagMap = getOrCreateTagMap()
val tagsElement = tooltipElement.selectFirst("div.message_main > p:contains(Thể loại)")?.ownText().orEmpty()
val mangaTags = tagsElement.split(',').mapNotNullToSet { tagMap[it.trim()] }
Manga(
id = generateUid(slug),
title = tooltipElement.selectFirst("div.title")?.text().orEmpty(),
altTitle = null,
url = absUrl.toRelativeUrl(getDomain()),
publicUrl = absUrl,
rating = RATING_UNKNOWN,
isNsfw = false,
coverUrl = item.selectFirst("div.image a img")?.absUrl("data-original").orEmpty(),
largeCoverUrl = null,
tags = mangaTags,
state = mangaState,
author = tooltipElement.selectFirst("div.message_main > p:contains(Tác giả)")?.ownText(),
description = tooltipElement.selectFirst("div.box_text")?.text(),
chapters = null,
source = source,
)
}
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val pageElements = context.httpGet(chapter.url.toAbsoluteUrl(getDomain())).parseHtml()
.select("div.reading-detail.box_doc > div img")
return pageElements.map { element ->
val url = element.attrAsAbsoluteUrl("data-original")
MangaPage(
id = generateUid(url),
url = url,
referer = getDomain(),
preview = null,
source = source,
)
}
}
override suspend fun getTags(): Set<MangaTag> {
val map = getOrCreateTagMap()
val tagSet = ArraySet<MangaTag>(map.size)
for (entry in map) {
tagSet.add(entry.value)
}
return tagSet
}
private suspend fun getOrCreateTagMap(): ArrayMap<String, MangaTag> = mutex.withLock {
tagCache?.let { return@withLock it }
val doc = context.httpGet("/tim-truyen-nang-cao".toAbsoluteUrl(getDomain())).parseHtml()
val tagItems = doc.select("div.genre-item")
val result = ArrayMap<String, MangaTag>(tagItems.size)
for (item in tagItems) {
val title = item.text().trim()
val key = item.select("span[data-id]").attr("data-id")
result[title] = MangaTag(title = title, key = key, source = source)
}
tagCache = result
result
}
private fun getSortOrderKey(sortOrder: SortOrder) = when (sortOrder) {
SortOrder.UPDATED -> 0
SortOrder.POPULARITY -> 10
SortOrder.NEWEST -> 15
SortOrder.RATING -> 20
else -> throw IllegalArgumentException("Sort order ${sortOrder.name} not supported")
}
}

View File

@@ -1,228 +0,0 @@
package org.koitharu.kotatsu.parsers.site
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaParser
import org.koitharu.kotatsu.parsers.MangaParserAuthProvider
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.exception.AuthRequiredException
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.text.SimpleDateFormat
import java.util.*
private const val MAX_THUMB_INDEX = 19
@MangaSourceParser("NUDEMOON", "Nude-Moon", "ru")
internal class NudeMoonParser(
override val context: MangaLoaderContext,
) : MangaParser(MangaSource.NUDEMOON), MangaParserAuthProvider {
override val configKeyDomain = ConfigKey.Domain("nude-moon.net", null)
override val authUrl: String
get() = "https://${getDomain()}/index.php"
override val isAuthorized: Boolean
get() {
return context.cookieJar.getCookies(getDomain()).any {
it.name == "fusion_user"
}
}
override val sortOrders: Set<SortOrder> = EnumSet.of(
SortOrder.NEWEST,
SortOrder.POPULARITY,
SortOrder.RATING,
)
init {
context.cookieJar.insertCookies(
getDomain(),
"NMfYa=1;",
"nm_mobile=0;",
)
}
override suspend fun getList(
offset: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val domain = getDomain()
val url = when {
!query.isNullOrEmpty() -> "https://$domain/search?stext=${query.urlEncoded()}&rowstart=$offset"
!tags.isNullOrEmpty() -> tags.joinToString(
separator = "_",
prefix = "https://$domain/tags/",
postfix = "&rowstart=$offset",
transform = { it.key.urlEncoded() },
)
else -> "https://$domain/all_manga?${getSortKey(sortOrder)}&rowstart=$offset"
}
val doc = context.httpGet(url).parseHtml()
val root = doc.body().run {
selectFirst("td.main-bg") ?: selectFirst("td.main-body")
} ?: doc.parseFailed("Cannot find root")
return root.select("table.news_pic2").mapNotNull { row ->
val a = row.selectFirst("td.bg_style1")?.selectFirst("a")
?: return@mapNotNull null
val href = a.attrAsRelativeUrl("href")
val title = a.selectFirst("h2")?.text().orEmpty()
val info = row.selectFirst("td[width=100%]") ?: return@mapNotNull null
Manga(
id = generateUid(href),
url = href,
title = title.substringAfter(" / "),
altTitle = title.substringBefore(" / ", "")
.takeUnless { it.isBlank() },
author = info.getElementsContainingOwnText("Автор:").firstOrNull()
?.nextElementSibling()?.ownText(),
coverUrl = row.selectFirst("img.news_pic2")?.absUrl("data-src")
.orEmpty(),
tags = row.selectFirst("span.tag-links")?.select("a")
?.mapToSet {
MangaTag(
title = it.text().toTitleCase(),
key = it.attr("href").substringAfterLast('/'),
source = source,
)
}.orEmpty(),
source = source,
publicUrl = a.absUrl("href"),
rating = RATING_UNKNOWN,
isNsfw = true,
description = row.selectFirst("div.description")?.html(),
state = null,
)
}
}
override suspend fun getDetails(manga: Manga): Manga {
val body = context.httpGet(manga.url.toAbsoluteUrl(getDomain())).parseHtml().body()
val root = body.selectFirst("table.shoutbox")
?: body.parseFailed("Cannot find root")
val info = root.select("div.tbl2")
val lastInfo = info.last()
return manga.copy(
largeCoverUrl = body.selectFirst("img.news_pic2")?.absUrl("src"),
description = info.select("div.blockquote").lastOrNull()?.html() ?: manga.description,
tags = info.select("span.tag-links").firstOrNull()?.select("a")?.mapToSet {
MangaTag(
title = it.text().toTitleCase(),
key = it.attr("href").substringAfterLast('/'),
source = source,
)
}?.plus(manga.tags) ?: manga.tags,
author = lastInfo?.getElementsByAttributeValueContaining("href", "mangaka/")?.text()
?: manga.author,
chapters = listOf(
MangaChapter(
id = manga.id,
url = getReadLink(manga.url),
source = source,
number = 1,
name = manga.title,
scanlator = lastInfo?.getElementsByAttributeValueContaining("href", "perevod/")?.text(),
uploadDate = lastInfo?.getElementsContainingOwnText("Дата:")
?.firstOrNull()
?.html()
?.parseDate() ?: 0L,
branch = null,
),
),
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(getDomain())
val doc = context.httpGet(fullUrl).parseHtml()
val mangaId = chapter.url.substringAfterLast('/').substringBefore('-').toIntOrNull()
val script = doc.select("script").firstNotNullOfOrNull {
it.html().takeIf { x -> x.contains(" images = new ") }
} ?: if (isAuthorized) {
doc.parseFailed("Cannot find pages list")
} else {
throw AuthRequiredException(source)
}
val pagesRegex = Regex("images\\[(\\d+)].src\\s*=\\s*'([^']+)'", RegexOption.MULTILINE)
return pagesRegex.findAll(script).map { match ->
val i = match.groupValues[1].toInt()
val url = match.groupValues[2]
MangaPage(
id = generateUid(url),
url = url,
referer = fullUrl,
preview = if (i <= MAX_THUMB_INDEX && mangaId != null) {
val part2 = url.substringBeforeLast('/')
val part3 = url.substringAfterLast('/')
val part1 = part2.substringBeforeLast('/')
"$part1/thumb/$mangaId/thumb_$part3"
} else {
null
},
source = source,
)
}.toList()
}
override suspend fun getTags(): Set<MangaTag> {
val domain = getDomain()
val doc = context.httpGet("https://$domain/all_manga").parseHtml()
val root = doc.body().getElementsContainingOwnText("Поиск манги по тегам")
.firstOrNull()?.parents()?.find { it.tag().normalName() == "tbody" }
?.selectFirst("td.textbox")?.selectFirst("td.small")
?: doc.parseFailed("Tags root not found")
return root.select("a").mapToSet {
MangaTag(
title = it.text().toTitleCase(),
key = it.attr("href").substringAfterLast('/')
.removeSuffix("+"),
source = source,
)
}
}
override suspend fun getUsername(): String {
val body = context.httpGet("https://${getDomain()}/").parseHtml()
.body()
return body
.getElementsContainingOwnText("Профиль")
.firstOrNull()
?.attr("href")
?.substringAfterLast('/')
?: run {
throw if (body.selectFirst("form[name=\"loginform\"]") != null) {
AuthRequiredException(source)
} else {
body.parseFailed("Cannot find username")
}
}
}
override fun getFaviconUrl(): String {
return "https://${getDomain()}/favicon.jpg"
}
private fun getSortKey(sortOrder: SortOrder) =
when (sortOrder) {
SortOrder.POPULARITY -> "views"
SortOrder.NEWEST -> "date"
SortOrder.RATING -> "like"
else -> "like"
}
private fun String.parseDate(): Long {
val dateString = substringBetweenFirst("Дата:", "<")?.trim() ?: return 0
val dateFormat = SimpleDateFormat("d MMMM yyyy", Locale("ru"))
return dateFormat.tryParse(dateString)
}
private fun getReadLink(url: String): String {
val prefix = url.substringBefore('-', "")
val suffix = url.substringAfter('-').trimStart('-')
return "$prefix-online-$suffix"
}
}

View File

@@ -1,177 +0,0 @@
package org.koitharu.kotatsu.parsers.site
import org.jsoup.nodes.Element
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import org.koitharu.kotatsu.parsers.util.json.getStringOrNull
import org.koitharu.kotatsu.parsers.util.json.mapJSON
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("UNION_MANGAS", "Union Mangás", "pt")
class UnionMangasParser(override val context: MangaLoaderContext) : PagedMangaParser(MangaSource.UNION_MANGAS, 40) {
override val sortOrders = EnumSet.of(
SortOrder.ALPHABETICAL,
SortOrder.POPULARITY,
)
override val configKeyDomain = ConfigKey.Domain("unionleitor.top", emptyArray())
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
if (!query.isNullOrEmpty()) {
return if (page == searchPaginator.firstPage) {
search(query)
} else {
emptyList()
}
}
val tag = tags.oneOrThrowIfMany()
val url = urlBuilder()
.addPathSegment("lista-mangas")
.addPathSegment(
when {
tag != null -> tag.key
sortOrder == SortOrder.ALPHABETICAL -> "a-z"
else -> "visualizacoes"
},
).addPathSegment(page.toString())
val doc = context.httpGet(url.build()).parseHtml()
val root = doc.selectFirstOrThrow("div.tamanho-bloco-perfil")
return root.select(".lista-mangas-novos").map { div ->
val a = div.selectFirstOrThrow("a")
val img = div.selectFirstOrThrow("img")
val href = a.attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = a.attrAsAbsoluteUrl("href"),
title = div.selectLastOrThrow("a").text(),
coverUrl = img.attrAsAbsoluteUrl("src"),
altTitle = null,
rating = RATING_UNKNOWN,
tags = emptySet(),
description = div.selectLast("div")?.ownText(),
state = null,
author = null,
isNsfw = false,
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga {
val doc = context.httpGet(manga.url.toAbsoluteUrl(getDomain())).parseHtml()
val root = doc.selectFirstOrThrow(".perfil-manga")
val dateFormat = SimpleDateFormat("dd/MM/yyyy", Locale.ROOT)
return manga.copy(
rating = root.select("h2")
.find { it.ownText().startsWith('#') }
?.ownText()?.drop(1)?.toFloatOrNull()?.div(10f) ?: manga.rating,
largeCoverUrl = root.selectFirst("img.img-thumbnail")?.attrAsAbsoluteUrlOrNull("src"),
description = root.selectFirst(".panel-default")?.selectFirst(".panel-body")?.html(),
author = root.tableValue("Autor")?.ownText(),
altTitle = root.tableValue("Título(s) Alternativo(s)")?.ownText(),
state = when (root.tableValue("Status")?.selectLast(".label")?.text()) {
"Completo" -> MangaState.FINISHED
"Ativo" -> MangaState.ONGOING
else -> null
},
tags = root.tableValue("Gênero(s)")?.select("a")?.mapToSet {
it.toMangaTag()
} ?: manga.tags,
isNsfw = root.selectFirst(".alert-danger")?.html()?.contains("18 anos") == true,
chapters = root.select("div.row.capitulos").asReversed().mapChapters { i, div ->
val a = div.selectFirstOrThrow("a")
val href = a.attrAsRelativeUrl("href")
val title = a.text()
MangaChapter(
id = generateUid(href),
name = title,
number = i + 1,
url = href,
scanlator = div.selectLast("a")?.text()?.takeUnless { it == title },
uploadDate = dateFormat.tryParse(
a.nextElementSibling()?.text()?.removeSurrounding('(', ')'),
),
branch = null,
source = source,
)
},
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(getDomain())
val doc = context.httpGet(fullUrl).parseHtml()
val root = doc.body().selectFirstOrThrow("article")
return root.selectOrThrow("img[pag]").mapNotNull { img ->
val href = img.attrAsRelativeUrl("src")
if (href.startsWith("/images/banner")) {
return@mapNotNull null
}
MangaPage(
id = generateUid(href),
url = href,
referer = fullUrl,
preview = null,
source = source,
)
}
}
override suspend fun getTags(): Set<MangaTag> {
val doc = context.httpGet(urlBuilder().addPathSegment("lista-mangas").build()).parseHtml()
val ul = doc.body().selectFirstOrThrow(".nav-tabs").selectFirstOrThrow("ul.dropdown-menu")
return ul.select("li").mapToSet { li ->
li.selectFirstOrThrow("a").toMangaTag()
}
}
private suspend fun search(query: String): List<Manga> {
val domain = getDomain()
val json = context.httpGet(
urlBuilder()
.addPathSegments("assets/busca.php")
.addQueryParameter("nomeManga", query)
.build(),
).parseJson()
return json.getJSONArray("items").mapJSON { jo ->
val href = "/pagina-manga/" + jo.getString("url")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(domain),
title = jo.getString("titulo"),
rating = RATING_UNKNOWN,
tags = emptySet(),
author = jo.getStringOrNull("autor"),
coverUrl = jo.getString("imagem"),
state = null,
isNsfw = false,
altTitle = null,
source = source,
)
}
}
private fun Element.tableValue(title: String): Element? {
return select("h4.media-heading")
.find { it.selectFirst("label.subtit-manga")?.text()?.contains(title, ignoreCase = true) == true }
}
private fun Element.toMangaTag() = MangaTag(
title = text().toTitleCase(sourceLocale ?: Locale.ROOT),
key = attr("href").removeSuffix('/').substringAfterLast('/'),
source = source,
)
}

View File

@@ -1,4 +1,4 @@
package org.koitharu.kotatsu.parsers.site
package org.koitharu.kotatsu.parsers.site.all
import androidx.collection.ArraySet
import org.json.JSONArray
@@ -11,6 +11,7 @@ import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.exception.ParseException
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import org.koitharu.kotatsu.parsers.util.json.isNullOrEmpty
import java.nio.charset.StandardCharsets
import java.security.MessageDigest
import java.util.*
@@ -19,58 +20,137 @@ import javax.crypto.spec.IvParameterSpec
import javax.crypto.spec.SecretKeySpec
@MangaSourceParser("BATOTO", "Bato.To")
internal class BatoToParser(override val context: MangaLoaderContext) : PagedMangaParser(
internal class BatoToParser(context: MangaLoaderContext) : PagedMangaParser(
context = context,
source = MangaSource.BATOTO,
pageSize = 60,
searchPageSize = 20,
) {
override val sortOrders: Set<SortOrder> = EnumSet.of(
override val availableSortOrders: Set<SortOrder> = EnumSet.of(
SortOrder.NEWEST,
SortOrder.UPDATED,
SortOrder.POPULARITY,
SortOrder.ALPHABETICAL,
)
override val availableStates: Set<MangaState> = EnumSet.allOf(MangaState::class.java)
override val isTagsExclusionSupported: Boolean = true
override val availableContentRating: Set<ContentRating> = EnumSet.of(ContentRating.SAFE)
override val configKeyDomain = ConfigKey.Domain(
"bato.to",
arrayOf("bato.to", "mto.to", "mangatoto.com", "battwo.com", "batotwo.com", "comiko.net", "batotoo.com"),
"batocomic.com",
"batocomic.net",
"batocomic.org",
"batotoo.com",
"batotwo.com",
"battwo.com",
"comiko.net",
"comiko.org",
"mangatoto.com",
"mangatoto.net",
"mangatoto.org",
"readtoto.com",
"readtoto.net",
"readtoto.org",
"dto.to",
"hto.to",
"mto.to",
"wto.to",
"xbato.com",
"xbato.net",
"xbato.org",
"zbato.com",
"zbato.net",
"zbato.org",
)
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
if (!query.isNullOrEmpty()) {
return search(page, query)
}
@Suppress("NON_EXHAUSTIVE_WHEN_STATEMENT")
val url = buildString {
append("https://")
append(getDomain())
append("/browse?sort=")
when (sortOrder) {
SortOrder.UPDATED,
-> append("update.za")
override suspend fun getListPage(page: Int, filter: MangaListFilter?): List<Manga> {
SortOrder.POPULARITY -> append("views_a.za")
SortOrder.NEWEST -> append("create.za")
SortOrder.ALPHABETICAL -> append("title.az")
when (filter) {
is MangaListFilter.Search -> {
return search(page, filter.query)
}
if (!tags.isNullOrEmpty()) {
append("&genres=")
appendAll(tags, ",") { it.key }
is MangaListFilter.Advanced -> {
val url = buildString {
append("https://")
append(domain)
append("/browse?sort=")
when (filter.sortOrder) {
SortOrder.UPDATED -> append("update.za")
SortOrder.POPULARITY -> append("views_a.za")
SortOrder.NEWEST -> append("create.za")
SortOrder.ALPHABETICAL -> append("title.az")
else -> append("update.za")
}
filter.states.oneOrThrowIfMany()?.let {
append("&release=")
append(
when (it) {
MangaState.ONGOING -> "ongoing"
MangaState.FINISHED -> "completed"
MangaState.ABANDONED -> "cancelled"
MangaState.PAUSED -> "hiatus"
MangaState.UPCOMING -> "pending"
},
)
}
filter.locale?.let {
append("&langs=")
append(it.language)
}
append("&genres=")
if (filter.tags.isNotEmpty()) {
appendAll(filter.tags, ",") { it.key }
}
append("|")
if (filter.tagsExclude.isNotEmpty()) {
appendAll(filter.tagsExclude, ",") { it.key }
}
if (filter.contentRating.isNotEmpty()) {
filter.contentRating.oneOrThrowIfMany()?.let {
append(
when (it) {
ContentRating.SAFE -> append(",gore,bloody,violence,ecchi,adult,mature,smut,hentai")
else -> append("")
},
)
}
}
append("&page=")
append(page.toString())
}
return parseList(url, page)
}
null -> {
val url = buildString {
append("https://")
append(domain)
append("/browse?sort=update.za")
append("&page=")
append(page.toString())
}
return parseList(url, page)
}
append("&page=")
append(page)
}
return parseList(url, page)
}
override suspend fun getDetails(manga: Manga): Manga {
val root = context.httpGet(manga.url.toAbsoluteUrl(getDomain())).parseHtml()
val root = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
.requireElementById("mainer")
val details = root.selectFirstOrThrow(".detail-set")
val attrs = details.selectFirst(".attr-main")?.select(".attr-item")?.associate {
@@ -84,28 +164,29 @@ internal class BatoToParser(override val context: MangaLoaderContext) : PagedMan
?.selectFirst(".limit-html")
?.html(),
tags = manga.tags + attrs["Genres:"]?.parseTags().orEmpty(),
state = when (attrs["Release status:"]?.text()) {
state = when (attrs["Original work:"]?.text()) {
"Ongoing" -> MangaState.ONGOING
"Completed" -> MangaState.FINISHED
"Cancelled" -> MangaState.ABANDONED
"Hiatus" -> MangaState.PAUSED
else -> manga.state
},
author = attrs["Authors:"]?.text()?.trim() ?: manga.author,
chapters = root.selectFirst(".episode-list")
?.selectFirst(".main")
?.children()
?.reversed()
?.mapChapters { i, div ->
?.mapChapters(reversed = true) { i, div ->
div.parseChapter(i)
}.orEmpty(),
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(getDomain())
val scripts = context.httpGet(fullUrl).parseHtml().select("script")
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val scripts = webClient.httpGet(fullUrl).parseHtml().select("script")
for (script in scripts) {
val scriptSrc = script.html()
val p = scriptSrc.indexOf("const imgHttpLis =")
val p = scriptSrc.indexOf("const imgHttps =")
if (p == -1) continue
val start = scriptSrc.indexOf('[', p)
val end = scriptSrc.indexOf(';', start)
@@ -125,8 +206,7 @@ internal class BatoToParser(override val context: MangaLoaderContext) : PagedMan
val url = images.getString(i)
result += MangaPage(
id = generateUid(url),
url = url + "?" + args.getString(i),
referer = fullUrl,
url = if (args.length() == 0) url else url + "?" + args.getString(i),
preview = null,
source = source,
)
@@ -136,9 +216,9 @@ internal class BatoToParser(override val context: MangaLoaderContext) : PagedMan
throw ParseException("Cannot find images list", fullUrl)
}
override suspend fun getTags(): Set<MangaTag> {
val scripts = context.httpGet(
"https://${getDomain()}/browse",
override suspend fun getAvailableTags(): Set<MangaTag> {
val scripts = webClient.httpGet(
"https://${domain}/browse",
).parseHtml().selectOrThrow("script")
for (script in scripts) {
val genres = script.html().substringBetweenFirst("const _genres =", ";") ?: continue
@@ -147,7 +227,7 @@ internal class BatoToParser(override val context: MangaLoaderContext) : PagedMan
jo.keys().forEach { key ->
val item = jo.getJSONObject(key)
result += MangaTag(
title = item.getString("text").toTitleCase(),
title = item.getString("text").toTitleCase(Locale.ENGLISH),
key = item.getString("file"),
source = source,
)
@@ -157,12 +237,26 @@ internal class BatoToParser(override val context: MangaLoaderContext) : PagedMan
throw ParseException("Cannot find gernes list", scripts[0].baseUri())
}
override fun getFaviconUrl(): String = "https://styles.amarkcdn.com/img/batoto/favicon.ico?v0"
override suspend fun getAvailableLocales(): Set<Locale> = setOf(
Locale.CHINESE, Locale.ENGLISH, Locale.US, Locale.FRENCH, Locale.GERMAN, Locale.ITALIAN, Locale.JAPANESE,
Locale("af"), Locale("ar"), Locale("az"), Locale("eu"), Locale("be"),
Locale("bn"), Locale("bs"), Locale("bg"), Locale("my"), Locale("km"),
Locale("ceb"), Locale("zh_hk"), Locale("zh_tw"), Locale("hr"), Locale("cs"),
Locale("da"), Locale("nl"), Locale("eo"), Locale("et"), Locale("fil"),
Locale("fi"), Locale("ka"), Locale("el"), Locale("ht"), Locale("he"),
Locale("hi"), Locale("hu"), Locale("id"), Locale("kk"), Locale("ko"),
Locale("lv"), Locale("ms"), Locale("ml"), Locale("mo"), Locale("mn"),
Locale("ne"), Locale("no"), Locale("fa"), Locale("pl"), Locale("pt"),
Locale("pt_br"), Locale("pt_pt"), Locale("ro"), Locale("ru"), Locale("sr"),
Locale("si"), Locale("sk"), Locale("es"), Locale("es_419"), Locale("ta"),
Locale("te"), Locale("th"), Locale("ti"), Locale("tr"), Locale("uk"),
Locale("vi"), Locale("zu"),
)
private suspend fun search(page: Int, query: String): List<Manga> {
val url = buildString {
append("https://")
append(getDomain())
append(domain)
append("/search?word=")
append(query.replace(' ', '+'))
append("&page=")
@@ -177,7 +271,7 @@ internal class BatoToParser(override val context: MangaLoaderContext) : PagedMan
?.toIntOrNull() ?: body.parseFailed("Cannot determine current page")
private suspend fun parseList(url: String, page: Int): List<Manga> {
val body = context.httpGet(url).parseHtml().body()
val body = webClient.httpGet(url).parseHtml().body()
if (body.selectFirst(".browse-no-matches") != null) {
return emptyList()
}
@@ -307,4 +401,4 @@ internal class BatoToParser(override val context: MangaLoaderContext) : PagedMan
} else byteArrayOf(),
)
}
}
}

View File

@@ -0,0 +1,262 @@
package org.koitharu.kotatsu.parsers.site.all
import androidx.collection.ArraySet
import androidx.collection.SparseArrayCompat
import org.json.JSONArray
import org.json.JSONObject
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import org.koitharu.kotatsu.parsers.util.json.*
import java.text.SimpleDateFormat
import java.util.*
/**
* cc
*/
private const val CHAPTERS_LIMIT = 99999
@MangaSourceParser("COMICK_FUN", "ComicK")
internal class ComickFunParser(context: MangaLoaderContext) : PagedMangaParser(context, MangaSource.COMICK_FUN, 20) {
override val configKeyDomain = ConfigKey.Domain("comick.cc")
override val availableSortOrders: Set<SortOrder> = EnumSet.of(
SortOrder.POPULARITY,
SortOrder.UPDATED,
SortOrder.RATING,
)
override val availableStates: Set<MangaState> =
EnumSet.of(MangaState.ONGOING, MangaState.FINISHED, MangaState.PAUSED, MangaState.ABANDONED)
private val tagsArray = SuspendLazy(::loadTags)
override suspend fun getListPage(page: Int, filter: MangaListFilter?): List<Manga> {
val domain = domain
val url = urlBuilder()
.host("api.$domain")
.addPathSegment("v1.0")
.addPathSegment("search")
.addQueryParameter("type", "comic")
.addQueryParameter("tachiyomi", "true")
.addQueryParameter("limit", pageSize.toString())
.addQueryParameter("page", page.toString())
when (filter) {
is MangaListFilter.Search -> {
url.addQueryParameter("q", filter.query)
}
null -> {
url.addQueryParameter("sort", "view")
}
is MangaListFilter.Advanced -> {
filter.tags.forEach { tag ->
url.addQueryParameter("genres", tag.key)
}
url.addQueryParameter(
"sort",
when (filter.sortOrder) {
SortOrder.POPULARITY -> "view"
SortOrder.RATING -> "rating"
else -> "uploaded"
},
)
filter.states.oneOrThrowIfMany()?.let {
url.addQueryParameter(
"status",
when (it) {
MangaState.ONGOING -> "1"
MangaState.FINISHED -> "2"
MangaState.ABANDONED -> "3"
MangaState.PAUSED -> "4"
else -> ""
},
)
}
}
}
val ja = webClient.httpGet(url.build()).parseJsonArray()
val tagsMap = tagsArray.get()
return ja.mapJSON { jo ->
val slug = jo.getString("slug")
Manga(
id = generateUid(slug),
title = jo.getString("title"),
altTitle = null,
url = slug,
publicUrl = "https://$domain/comic/$slug",
rating = jo.getDoubleOrDefault("rating", -10.0).toFloat() / 10f,
isNsfw = false,
coverUrl = jo.getString("cover_url"),
largeCoverUrl = null,
description = jo.getStringOrNull("desc"),
tags = jo.selectGenres(tagsMap),
state = when (jo.getIntOrDefault("status", 0)) {
1 -> MangaState.ONGOING
2 -> MangaState.FINISHED
3 -> MangaState.ABANDONED
4 -> MangaState.PAUSED
else -> null
},
author = null,
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga {
val domain = domain
val url = "https://api.$domain/comic/${manga.url}?tachiyomi=true"
val jo = webClient.httpGet(url).parseJson()
val comic = jo.getJSONObject("comic")
var alt = ""
comic.getJSONArray("md_titles").mapJSON { alt += it.getString("title") + " - " }
return manga.copy(
altTitle = alt.ifEmpty { comic.getStringOrNull("title") },
isNsfw = jo.getBoolean("matureContent") || comic.getBoolean("hentai"),
description = comic.getStringOrNull("parsed") ?: comic.getStringOrNull("desc"),
tags = manga.tags + comic.getJSONArray("md_comic_md_genres").mapJSONToSet {
val g = it.getJSONObject("md_genres")
MangaTag(
title = g.getString("name"),
key = g.getString("slug"),
source = source,
)
},
author = jo.getJSONArray("artists").optJSONObject(0)?.getString("name"),
chapters = getChapters(comic.getString("hid")),
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val jo = webClient.httpGet(
"https://api.${domain}/chapter/${chapter.url}?tachiyomi=true",
).parseJson().getJSONObject("chapter")
return jo.getJSONArray("images").mapJSON {
val url = it.getString("url")
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
val sparseArray = tagsArray.get()
val set = ArraySet<MangaTag>(sparseArray.size())
for (i in 0 until sparseArray.size()) {
set.add(sparseArray.valueAt(i))
}
return set
}
private suspend fun loadTags(): SparseArrayCompat<MangaTag> {
val ja = webClient.httpGet("https://api.${domain}/genre").parseJsonArray()
val tags = SparseArrayCompat<MangaTag>(ja.length())
for (jo in ja.JSONIterator()) {
tags.append(
jo.getInt("id"),
MangaTag(
title = jo.getString("name").toTitleCase(Locale.ENGLISH),
key = jo.getString("slug"),
source = source,
),
)
}
return tags
}
private suspend fun getChapters(hid: String): List<MangaChapter> {
val ja = webClient.httpGet(
url = "https://api.${domain}/comic/$hid/chapters?limit=$CHAPTERS_LIMIT",
).parseJson().getJSONArray("chapters")
val dateFormat = SimpleDateFormat("yyyy-MM-dd")
val counters = HashMap<String?, Int>()
return ja.toJSONList().reversed().mapChapters { _, jo ->
val vol = jo.getStringOrNull("vol")
val chap = jo.getStringOrNull("chap")
val locale = Locale.forLanguageTag(jo.getString("lang"))
val group = jo.optJSONArray("group_name")?.joinToString(", ")
val branch = buildString {
append(locale.getDisplayName(locale).toTitleCase(locale))
if (!group.isNullOrEmpty()) {
append(" (")
append(group)
append(')')
}
}
MangaChapter(
id = generateUid(jo.getLong("id")),
name = buildString {
vol?.let { append("Vol ").append(it).append(' ') }
chap?.let { append("Chap ").append(it) }
jo.getStringOrNull("title")?.let { append(": ").append(it) }
},
number = counters.incrementAndGet(branch),
url = jo.getString("hid"),
scanlator = jo.optJSONArray("group_name")?.asIterable<String>()?.joinToString()
?.takeUnless { it.isBlank() },
uploadDate = dateFormat.tryParse(jo.getString("created_at").substringBefore('T')),
branch = branch,
source = source,
)
}
/*val chaptersBuilder = ChaptersListBuilder(list.size)
val branchedChapters = HashMap<String?, HashMap<Pair<String?, String?>, MangaChapter>>()
for (jo in list) {
val vol = jo.getStringOrNull("vol")
val chap = jo.getStringOrNull("chap")
val volChap = vol to chap
val locale = Locale.forLanguageTag(jo.getString("lang"))
val lc = locale.getDisplayName(locale).toTitleCase(locale)
val branch = (list.indices).firstNotNullOf { i ->
val b = if (i == 0) lc else "$lc ($i)"
if (branchedChapters[b]?.get(volChap) == null) b else null
}
val chapter = MangaChapter(
id = generateUid(jo.getLong("id")),
name = buildString {
vol?.let { append("Vol ").append(it).append(' ') }
chap?.let { append("Chap ").append(it) }
jo.getStringOrNull("title")?.let { append(": ").append(it) }
},
number = branchedChapters[branch]?.size?.plus(1) ?: 1,
url = jo.getString("hid"),
scanlator = jo.optJSONArray("group_name")?.asIterable<String>()?.joinToString()
?.takeUnless { it.isBlank() },
uploadDate = dateFormat.tryParse(jo.getString("created_at").substringBefore('T')),
branch = branch,
source = source,
)
if (chaptersBuilder.add(chapter)) {
branchedChapters.getOrPut(branch, ::HashMap)[volChap] = chapter
}
}
return chaptersBuilder.toList()*/
}
private fun JSONObject.selectGenres(tags: SparseArrayCompat<MangaTag>): Set<MangaTag> {
val array = optJSONArray("genres") ?: return emptySet()
val res = ArraySet<MangaTag>(array.length())
for (i in 0 until array.length()) {
val id = array.getInt(i)
val tag = tags[id] ?: continue
res.add(tag)
}
return res
}
private fun JSONArray.joinToString(separator: String): String {
return (0 until length()).joinToString(separator) { i -> getString(i) }
}
}

View File

@@ -0,0 +1,420 @@
package org.koitharu.kotatsu.parsers.site.all
import androidx.collection.ArrayMap
import androidx.collection.ArraySet
import androidx.collection.SparseArrayCompat
import androidx.collection.set
import okhttp3.HttpUrl.Companion.toHttpUrlOrNull
import org.jsoup.internal.StringUtil
import org.jsoup.nodes.Element
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaParserAuthProvider
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.exception.AuthRequiredException
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.util.*
import kotlin.math.pow
private const val DOMAIN_UNAUTHORIZED = "e-hentai.org"
private const val DOMAIN_AUTHORIZED = "exhentai.org"
@MangaSourceParser("EXHENTAI", "ExHentai", type = ContentType.HENTAI)
internal class ExHentaiParser(
context: MangaLoaderContext,
) : PagedMangaParser(context, MangaSource.EXHENTAI, pageSize = 25), MangaParserAuthProvider {
override val availableSortOrders: Set<SortOrder> = setOf(SortOrder.NEWEST)
override val isTagsExclusionSupported: Boolean = true
override val configKeyDomain: ConfigKey.Domain
get() = ConfigKey.Domain(
if (isAuthorized) DOMAIN_AUTHORIZED else DOMAIN_UNAUTHORIZED,
if (isAuthorized) DOMAIN_UNAUTHORIZED else DOMAIN_AUTHORIZED,
)
override val authUrl: String
get() = "https://${domain}/bounce_login.php"
private val ratingPattern = Regex("-?[0-9]+px")
private val authCookies = arrayOf("ipb_member_id", "ipb_pass_hash")
private var updateDm = false
private val nextPages = SparseArrayCompat<Long>()
private val suspiciousContentKey = ConfigKey.ShowSuspiciousContent(false)
private val tagsMap = SuspendLazy(::fetchTags)
override val isAuthorized: Boolean
get() {
val authorized = isAuthorized(DOMAIN_UNAUTHORIZED)
if (authorized) {
if (!isAuthorized(DOMAIN_AUTHORIZED)) {
context.cookieJar.copyCookies(
DOMAIN_UNAUTHORIZED,
DOMAIN_AUTHORIZED,
authCookies,
)
context.cookieJar.insertCookies(DOMAIN_AUTHORIZED, "yay=louder")
}
return true
}
return false
}
init {
context.cookieJar.insertCookies(DOMAIN_AUTHORIZED, "nw=1", "sl=dm_2")
context.cookieJar.insertCookies(DOMAIN_UNAUTHORIZED, "nw=1", "sl=dm_2")
paginator.firstPage = 0
}
override suspend fun getListPage(page: Int, filter: MangaListFilter?): List<Manga> {
val next = nextPages.get(page, 0L)
if (page > 0 && next == 0L) {
assert(false) { "Page timestamp not found" }
return emptyList()
}
var search = ""
val url = buildString {
append("https://")
append(domain)
append("/?next=")
append(next)
when (filter) {
is MangaListFilter.Search -> {
search += filter.query.urlEncoded()
append("&f_search=")
append(search.trim().replace(' ', '+'))
}
is MangaListFilter.Advanced -> {
filter.toSearchQuery()?.let { sq ->
append("&f_search=")
append(sq.urlEncoded())
}
val catsOn = filter.tags.mapNotNullToSet { it.key.toIntOrNull() }
val catsOff = filter.tagsExclude.mapNotNullToSet { it.key.toIntOrNull() }
if (catsOff.size >= 10) {
return emptyList()
}
var fCats = catsOn.fold(0, Int::or)
if (fCats != 0) {
fCats = 1023 - fCats
}
fCats = catsOff.fold(fCats, Int::or)
if (fCats != 0) {
append("&f_cats=")
append(fCats)
}
}
null -> {}
}
// by unknown reason cookie "sl=dm_2" is ignored, so, we should request it again
if (updateDm) {
append("&inline_set=dm_e")
}
append("&advsearch=1")
if (config[suspiciousContentKey]) {
append("&f_sh=on")
}
}
val body = webClient.httpGet(url).parseHtml().body()
val root = body.selectFirst("table.itg")
?.selectFirst("tbody")
?: if (updateDm) {
body.parseFailed("Cannot find root")
} else {
updateDm = true
return getListPage(page, filter)
}
updateDm = false
nextPages[page + 1] = getNextTimestamp(body)
return root.children().mapNotNull { tr ->
if (tr.childrenSize() != 2) return@mapNotNull null
val (td1, td2) = tr.children()
val gLink = td2.selectFirstOrThrow("div.glink")
val a = gLink.parents().select("a").first() ?: gLink.parseFailed("link not found")
val href = a.attrAsRelativeUrl("href")
val tagsDiv = gLink.nextElementSibling() ?: gLink.parseFailed("tags div not found")
val mainTag = td2.selectFirst("div.cn")?.let { div ->
MangaTag(
title = div.text().toTitleCase(),
key = tagIdByClass(div.classNames()) ?: return@let null,
source = source,
)
}
Manga(
id = generateUid(href),
title = gLink.text().cleanupTitle(),
altTitle = null,
url = href,
publicUrl = a.absUrl("href"),
rating = td2.selectFirst("div.ir")?.parseRating() ?: RATING_UNKNOWN,
isNsfw = true,
coverUrl = td1.selectFirst("img")?.absUrl("src").orEmpty(),
tags = setOfNotNull(mainTag),
state = null,
author = tagsDiv.getElementsContainingOwnText("artist:").first()
?.nextElementSibling()?.text(),
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga {
val doc = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
val root = doc.body().selectFirstOrThrow("div.gm")
val cover = root.getElementById("gd1")?.children()?.first()
val title = root.getElementById("gd2")
val tagList = root.getElementById("taglist")
val tabs = doc.body().selectFirst("table.ptt")?.selectFirst("tr")
val lang = root.getElementById("gd3")
?.selectFirst("tr:contains(Language)")
?.selectFirst(".gdt2")?.ownTextOrNull()
val tagMap = tagsMap.get()
val tags = ArraySet<MangaTag>()
tagList?.selectFirst("tr:contains(female:)")?.select("a")?.mapNotNullTo(tags) { tagMap[it.text()] }
tagList?.selectFirst("tr:contains(male:)")?.select("a")?.mapNotNullTo(tags) { tagMap[it.text()] }
return manga.copy(
title = title?.getElementById("gn")?.text()?.cleanupTitle() ?: manga.title,
altTitle = title?.getElementById("gj")?.text()?.cleanupTitle() ?: manga.altTitle,
publicUrl = doc.baseUri().ifEmpty { manga.publicUrl },
rating = root.getElementById("rating_label")?.text()
?.substringAfterLast(' ')
?.toFloatOrNull()
?.div(5f) ?: manga.rating,
largeCoverUrl = cover?.styleValueOrNull("background")?.cssUrl(),
tags = tags,
description = tagList?.select("tr")?.joinToString("<br>") { tr ->
val (tc, td) = tr.children()
val subTags = td.select("a").joinToString { it.html() }
"<b>${tc.html()}</b> $subTags"
},
chapters = tabs?.select("a")?.findLast { a ->
a.text().toIntOrNull() != null
}?.let { a ->
val count = a.text().toInt()
val chapters = ChaptersListBuilder(count)
for (i in 1..count) {
val url = "${manga.url}?p=${i - 1}"
chapters += MangaChapter(
id = generateUid(url),
name = "${manga.title} #$i",
number = i,
url = url,
uploadDate = 0L,
source = source,
scanlator = null,
branch = lang,
)
}
chapters.toList()
},
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val doc = webClient.httpGet(chapter.url.toAbsoluteUrl(domain)).parseHtml()
val root = doc.body().requireElementById("gdt")
return root.select("a").map { a ->
val url = a.attrAsRelativeUrl("href")
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
)
}
}
override suspend fun getPageUrl(page: MangaPage): String {
val doc = webClient.httpGet(page.url.toAbsoluteUrl(domain)).parseHtml()
return doc.body().requireElementById("img").attrAsAbsoluteUrl("src")
}
private val tags =
"ahegao,anal,angel,apron,bandages,bbw,bdsm,beauty mark,big areolae,big ass,big breasts,big clit,big lips," +
"big nipples,bikini,blackmail,bloomers,blowjob,bodysuit,bondage,breast expansion,bukkake,bunny girl,business suit," +
"catgirl,centaur,cheating,chinese dress,christmas,collar,corset,cosplaying,cowgirl,crossdressing,cunnilingus," +
"dark skin,daughter,deepthroat,defloration,demon girl,double penetration,dougi,dragon,drunk,elf,exhibitionism,farting," +
"females only,femdom,filming,fingering,fishnets,footjob,fox girl,furry,futanari,garter belt,ghost,giantess," +
"glasses,gloves,goblin,gothic lolita,growth,guro,gyaru,hair buns,hairy,hairy armpits,handjob,harem,hidden sex," +
"horns,huge breasts,humiliation,impregnation,incest,inverted nipples,kemonomimi,kimono,kissing,lactation," +
"latex,leg lock,leotard,lingerie,lizard girl,maid,masked face,masturbation,midget,miko,milf,mind break," +
"mind control,monster girl,mother,muscle,nakadashi,netorare,nose hook,nun,nurse,oil,paizuri,panda girl," +
"pantyhose,piercing,pixie cut,policewoman,ponytail,pregnant,rape,rimjob,robot,scat,lolicon,schoolgirl uniform," +
"sex toys,shemale,sister,small breasts,smell,sole dickgirl,sole female,squirting,stockings,sundress,sweating," +
"swimsuit,swinging,tail,tall girl,teacher,tentacles,thigh high boots,tomboy,transformation,twins,twintails," +
"unusual pupils,urination,vore,vtuber,widow,wings,witch,wolf girl,x-ray,yuri,zombie,sole male,males only,yaoi," +
"tomgirl,tall man,oni,shotacon,prostate massage,policeman,males only,huge penis,fox boy,feminization,dog boy,dickgirl on male,big penis"
override suspend fun getAvailableTags(): Set<MangaTag> {
return tagsMap.get().values.toSet()
}
private suspend fun fetchTags(): Map<String, MangaTag> {
val tagMap = ArrayMap<String, MangaTag>()
val tagElements = tags.split(",")
for (el in tagElements) {
if (el.isEmpty()) continue
tagMap[el] = MangaTag(
title = el.toTitleCase(Locale.ENGLISH),
key = el,
source = source,
)
}
val doc = webClient.httpGet("https://${domain}").parseHtml()
val root = doc.body().requireElementById("searchbox").selectFirstOrThrow("table")
root.select("div.cs").mapNotNullToSet { div ->
val id = div.id().substringAfterLast('_').toIntOrNull() ?: return@mapNotNullToSet null
val name = div.text().toTitleCase(Locale.ENGLISH)
tagMap[name] = MangaTag(
title = "Kind: $name",
key = id.toString(),
source = source,
)
}
return tagMap
}
override suspend fun getAvailableLocales(): Set<Locale> = setOf(
Locale.JAPANESE,
Locale.ENGLISH,
Locale.CHINESE,
Locale("nl"),
Locale.FRENCH,
Locale.GERMAN,
Locale("hu"),
Locale.ITALIAN,
Locale("kr"),
Locale("pl"),
Locale("pt"),
Locale("ru"),
Locale("es"),
Locale("th"),
Locale("vi"),
)
private fun Locale.toLanguagePath() = when (language) {
else -> getDisplayLanguage(Locale.ENGLISH).lowercase()
}
override suspend fun getUsername(): String {
val doc = webClient.httpGet("https://forums.$DOMAIN_UNAUTHORIZED/").parseHtml().body()
val username = doc.getElementById("userlinks")
?.getElementsByAttributeValueContaining("href", "showuser=")
?.firstOrNull()
?.ownText()
?: if (doc.getElementById("userlinksguest") != null) {
throw AuthRequiredException(source)
} else {
doc.parseFailed()
}
return username
}
override fun onCreateConfig(keys: MutableCollection<ConfigKey<*>>) {
super.onCreateConfig(keys)
keys.add(suspiciousContentKey)
}
private fun isAuthorized(domain: String): Boolean {
val cookies = context.cookieJar.getCookies(domain).mapToSet { x -> x.name }
return authCookies.all { it in cookies }
}
private fun Element.parseRating(): Float {
return runCatching {
val style = requireNotNull(attr("style"))
val (v1, v2) = ratingPattern.find(style)!!.destructured
var p1 = v1.dropLast(2).toInt()
val p2 = v2.dropLast(2).toInt()
if (p2 != -1) {
p1 += 8
}
(80 - p1) / 80f
}.getOrDefault(RATING_UNKNOWN)
}
private fun String.cleanupTitle(): String {
val result = StringBuilder(length)
var skip = false
for (c in this) {
when {
c == '[' -> skip = true
c == ']' -> skip = false
c.isWhitespace() && result.isEmpty() -> continue
!skip -> result.append(c)
}
}
while (result.lastOrNull()?.isWhitespace() == true) {
result.deleteCharAt(result.lastIndex)
}
return result.toString()
}
private fun String.cssUrl(): String? {
val fromIndex = indexOf("url(")
if (fromIndex == -1) {
return null
}
val toIndex = indexOf(')', startIndex = fromIndex)
return if (toIndex == -1) {
null
} else {
substring(fromIndex + 4, toIndex).trim()
}
}
private fun tagIdByClass(classNames: Collection<String>): String? {
val className = classNames.find { x -> x.startsWith("ct") } ?: return null
val num = className.drop(2).toIntOrNull(16) ?: return null
return 2.0.pow(num).toInt().toString()
}
private fun getNextTimestamp(root: Element): Long {
return root.getElementById("unext")
?.attrAsAbsoluteUrlOrNull("href")
?.toHttpUrlOrNull()
?.queryParameter("next")
?.toLongOrNull() ?: 1
}
private fun MangaListFilter.Advanced.toSearchQuery(): String? {
val joiner = StringUtil.StringJoiner(" ")
for (tag in tags) {
if (tag.key.isNumeric()) {
continue
}
joiner.add("tag:\"")
joiner.append(tag.key)
joiner.append("\"$")
}
for (tag in tagsExclude) {
if (tag.key.isNumeric()) {
continue
}
joiner.add("-tag:\"")
joiner.append(tag.key)
joiner.append("\"$")
}
locale?.let { lc ->
joiner.add("language:\"")
joiner.append(lc.toLanguagePath())
joiner.append("\"$")
}
return joiner.complete().takeUnless { it.isEmpty() }
}
}

View File

@@ -0,0 +1,706 @@
package org.koitharu.kotatsu.parsers.site.all
import androidx.collection.ArraySet
import kotlinx.coroutines.async
import kotlinx.coroutines.awaitAll
import kotlinx.coroutines.coroutineScope
import kotlinx.coroutines.sync.Mutex
import kotlinx.coroutines.sync.withLock
import okhttp3.Headers
import org.json.JSONArray
import org.json.JSONObject
import org.jsoup.Jsoup
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaParser
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import org.koitharu.kotatsu.parsers.util.json.getStringOrNull
import org.koitharu.kotatsu.parsers.util.json.mapJSON
import java.nio.ByteBuffer
import java.nio.ByteOrder
import java.security.MessageDigest
import java.text.SimpleDateFormat
import java.util.*
import kotlin.math.min
@OptIn(ExperimentalUnsignedTypes::class)
@MangaSourceParser("HITOMILA", "Hitomi.La", type = ContentType.HENTAI)
class HitomiLaParser(context: MangaLoaderContext) : MangaParser(context, MangaSource.HITOMILA) {
override val configKeyDomain = ConfigKey.Domain("hitomi.la")
private val ltnBaseUrl get() = "https://${getDomain("ltn")}"
override val availableSortOrders: Set<SortOrder> = EnumSet.of(
SortOrder.NEWEST,
SortOrder.POPULARITY,
)
private val localeMap: Map<Locale, String> = mapOf(
Locale("id") to "indonesian",
Locale("jv") to "javanese",
Locale("ca") to "catalan",
Locale("ceb") to "cebuano",
Locale("cs") to "czech",
Locale("da") to "danish",
Locale("de") to "german",
Locale("et") to "estonian",
Locale.ENGLISH to "english",
Locale("es") to "spanish",
Locale("eo") to "esperanto",
Locale("fr") to "french",
Locale("it") to "italian",
Locale("hi") to "hindi",
Locale("hu") to "hungarian",
Locale("pl") to "polish",
Locale("pt") to "portuguese",
Locale("vi") to "vietnamese",
Locale("tr") to "turkish",
Locale("ru") to "russian",
Locale("uk") to "ukrainian",
Locale("ar") to "arabic",
Locale.KOREAN to "korean",
Locale.CHINESE to "chinese",
Locale.JAPANESE to "japanese",
)
private fun Locale?.getSiteLang(): String = when (this) {
null -> "all"
else -> localeMap[this] ?: "all"
}
override suspend fun getAvailableLocales(): Set<Locale> = localeMap.keys
override suspend fun getAvailableTags(): Set<MangaTag> = coroutineScope {
('a'..'z').map { alphabet ->
async {
val doc = webClient.httpGet("https://$domain/alltags-$alphabet.html").parseHtml()
doc.select(".posts > li").mapNotNull { element ->
val num =
element.ownText().let {
Regex("""\((\d+)\)""").find(it)?.groupValues?.get(1)?.toIntOrNull() ?: 0
}
if (num > 100) {
val url = element.selectFirst("a")
val href =
url?.attrAsRelativeUrl("href")
?: return@mapNotNull null
MangaTag(
title = url.ownText().toTagTitle(),
key = href.tagUrlToTag(),
source = source,
)
} else {
null
}
}
}
}.awaitAll().flatten().toSet()
}
private var cachedSearchIds: List<Int> = emptyList()
override suspend fun getList(
offset: Int,
filter: MangaListFilter?,
): List<Manga> = when (filter) {
is MangaListFilter.Advanced -> {
if (filter.tags.isEmpty()) {
when (filter.sortOrder) {
SortOrder.POPULARITY -> {
getGalleryIDsFromNozomi(
"popular",
"today",
filter.locale.getSiteLang(),
offset.nextOffsetRange(),
)
}
else -> {
getGalleryIDsFromNozomi(null, "index", filter.locale.getSiteLang(), offset.nextOffsetRange())
}
}
} else {
if (offset == 0) {
cachedSearchIds =
hitomiSearch(
filter.tags.joinToString(" ") { it.key },
filter.sortOrder == SortOrder.POPULARITY,
filter.locale.getSiteLang(),
).toList()
}
cachedSearchIds.subList(offset, min(offset + 25, cachedSearchIds.size))
}
}
is MangaListFilter.Search -> {
if (offset == 0) {
cachedSearchIds = hitomiSearch(filter.query, filter.sortOrder == SortOrder.POPULARITY).toList()
}
cachedSearchIds.subList(offset, min(offset + 25, cachedSearchIds.size))
}
else -> getGalleryIDsFromNozomi(null, "popular", "all", offset.nextOffsetRange())
}.toMangaList()
private fun Int.nextOffsetRange(): LongRange {
val bytes = this * 4L
return bytes.until(bytes + 100L)
}
private suspend fun hitomiSearch(
query: String,
sortByPopularity: Boolean = false,
language: String = "all",
): Set<Int> =
coroutineScope {
val terms = query
.trim()
.replace(Regex("""^\?"""), "")
.lowercase()
.split(Regex("\\s+"))
.map {
it.replace('_', ' ')
}
val positiveTerms = LinkedList<String>()
val negativeTerms = LinkedList<String>()
for (term in terms) {
if (term.startsWith("-")) {
negativeTerms.push(term.removePrefix("-"))
} else if (term.isNotBlank()) {
positiveTerms.push(term)
}
}
val positiveResults = positiveTerms.map {
async {
runCatchingCancellable {
getGalleryIDsForQuery(it, language)
}.getOrDefault(emptySet())
}
}
val negativeResults = negativeTerms.map {
async {
runCatchingCancellable {
getGalleryIDsForQuery(it, language)
}.getOrDefault(emptySet())
}
}
val results = when {
sortByPopularity -> getGalleryIDsFromNozomi(null, "popular", language)
positiveTerms.isEmpty() -> getGalleryIDsFromNozomi(null, "index", language)
else -> emptySet()
}.toMutableSet()
fun filterPositive(newResults: Set<Int>) {
when {
results.isEmpty() -> results.addAll(newResults)
else -> results.retainAll(newResults)
}
}
fun filterNegative(newResults: Set<Int>) {
results.removeAll(newResults)
}
// positive results
positiveResults.forEach {
filterPositive(it.await())
}
// negative results
negativeResults.forEach {
filterNegative(it.await())
}
results
}
// search.js
private suspend fun getGalleryIDsForQuery(
query: String,
language: String = "all",
): Set<Int> {
query.replace("_", " ").let {
if (it.indexOf(':') > -1) {
val sides = it.split(":")
val ns = sides[0]
var tag = sides[1]
var area: String? = ns
var lang = language
when (ns) {
"female", "male" -> {
area = "tag"
tag = it
}
"language" -> {
area = null
lang = tag
tag = "index"
}
}
return getGalleryIDsFromNozomi(area, tag, lang)
}
val key = hashTerm(it)
val node = getGalleryNodeAtAddress(0)
val data = bSearch(key, node) ?: return emptySet()
return getGalleryIDsFromData(data)
}
}
private suspend fun getGalleryIDsFromData(data: Pair<Long, Int>): Set<Int> {
val url = "$ltnBaseUrl/galleriesindex/galleries.${galleriesIndexVersion.get()}.data"
val (offset, length) = data
require(length in 1..100000000) {
"Length $length is too long"
}
val inbuf = getRangedResponse(url, offset.until(offset + length))
val galleryIDs = mutableSetOf<Int>()
val buffer =
ByteBuffer
.wrap(inbuf)
.order(ByteOrder.BIG_ENDIAN)
val numberOfGalleryIDs = buffer.int
val expectedLength = numberOfGalleryIDs * 4 + 4
require(numberOfGalleryIDs in 1..10000000) {
"number_of_galleryids $numberOfGalleryIDs is too long"
}
require(inbuf.size == expectedLength) {
"inbuf.byteLength ${inbuf.size} != expected_length $expectedLength"
}
for (i in 0.until(numberOfGalleryIDs))
galleryIDs.add(buffer.int)
return galleryIDs
}
private suspend fun bSearch(
key: UByteArray,
node: Node,
): Pair<Long, Int>? {
fun compareArrayBuffers(
dv1: UByteArray,
dv2: UByteArray,
): Int {
val top = min(dv1.size, dv2.size)
for (i in 0.until(top)) {
if (dv1[i] < dv2[i]) {
return -1
} else if (dv1[i] > dv2[i]) {
return 1
}
}
return 0
}
fun locateKey(
key: UByteArray,
node: Node,
): Pair<Boolean, Int> {
for (i in node.keys.indices) {
val cmpResult = compareArrayBuffers(key, node.keys[i])
if (cmpResult <= 0) {
return Pair(cmpResult == 0, i)
}
}
return Pair(false, node.keys.size)
}
fun isLeaf(node: Node): Boolean {
for (subnode in node.subNodeAddresses)
if (subnode != 0L) {
return false
}
return true
}
if (node.keys.isEmpty()) {
return null
}
val (there, where) = locateKey(key, node)
if (there) {
return node.datas[where]
} else if (isLeaf(node)) {
return null
}
val nextNode = getGalleryNodeAtAddress(node.subNodeAddresses[where])
return bSearch(key, nextNode)
}
private suspend fun getGalleryIDsFromNozomi(
area: String?,
tag: String,
language: String,
range: LongRange? = null,
): Set<Int> {
val nozomiAddress = when (area) {
null -> "$ltnBaseUrl/$tag-$language.nozomi"
else -> "$ltnBaseUrl/$area/$tag-$language.nozomi"
}
val bytes = getRangedResponse(nozomiAddress, range)
val nozomi = mutableSetOf<Int>()
val arrayBuffer = ByteBuffer
.wrap(bytes)
.order(ByteOrder.BIG_ENDIAN)
while (arrayBuffer.hasRemaining())
nozomi.add(arrayBuffer.int)
return nozomi
}
private val galleriesIndexVersion = SuspendLazy {
webClient.httpGet("$ltnBaseUrl/galleriesindex/version?_=${System.currentTimeMillis()}").parseRaw()
}
private data class Node(
val keys: List<UByteArray>,
val datas: List<Pair<Long, Int>>,
val subNodeAddresses: List<Long>,
)
private fun decodeNode(data: ByteArray): Node {
val buffer = ByteBuffer
.wrap(data)
.order(ByteOrder.BIG_ENDIAN)
val uData = data.toUByteArray()
val numberOfKeys = buffer.int
val keys = ArrayList<UByteArray>()
for (i in 0.until(numberOfKeys)) {
val keySize = buffer.int
if (keySize == 0 || keySize > 32) {
throw Exception("fatal: !keySize || keySize > 32")
}
keys.add(uData.sliceArray(buffer.position().until(buffer.position() + keySize)))
buffer.position(buffer.position() + keySize)
}
val numberOfDatas = buffer.int
val datas = ArrayList<Pair<Long, Int>>()
for (i in 0.until(numberOfDatas)) {
val offset = buffer.long
val length = buffer.int
datas.add(Pair(offset, length))
}
val numberOfSubNodeAddresses = 16 + 1
val subNodeAddresses = ArrayList<Long>()
for (i in 0.until(numberOfSubNodeAddresses)) {
val subNodeAddress = buffer.long
subNodeAddresses.add(subNodeAddress)
}
return Node(keys, datas, subNodeAddresses)
}
private suspend fun getGalleryNodeAtAddress(address: Long): Node {
val url = "$ltnBaseUrl/galleriesindex/galleries.${galleriesIndexVersion.get()}.index"
val nodedata = getRangedResponse(url, address.until(address + 464))
return decodeNode(nodedata)
}
private suspend fun getRangedResponse(
url: String,
range: LongRange? = null,
): ByteArray {
val rangeHeaders = when (range) {
null -> Headers.headersOf()
else -> Headers.headersOf("Range", "bytes=${range.first}-${range.last}")
}
return webClient.httpGet(url, rangeHeaders).parseBytes()
}
private fun hashTerm(term: String): UByteArray {
return sha256(term.toByteArray()).copyOfRange(0, 4).toUByteArray()
}
private fun sha256(data: ByteArray): ByteArray {
return MessageDigest.getInstance("SHA-256").digest(data)
}
private suspend fun Collection<Int>.toMangaList(): List<Manga> = coroutineScope {
map { id ->
async {
runCatchingCancellable {
val doc = webClient.httpGet("$ltnBaseUrl/galleryblock/$id.html").let {
val baseUri = it.request.url.toString()
val html = it.parseRaw()
Jsoup.parse(rewriteTnPaths(html), baseUri)
}
Manga(
id = generateUid(id.toString()),
title = doc.selectFirstOrThrow("h1").text(),
url = id.toString(),
coverUrl =
"https:" +
doc.selectFirstOrThrow("picture > source")
.attr("data-srcset")
.substringBefore(" "),
publicUrl =
doc.selectFirstOrThrow("h1 > a")
.attrAsRelativeUrl("href")
.toAbsoluteUrl(domain),
author = null,
tags = emptySet(),
isNsfw = true,
rating = RATING_UNKNOWN,
altTitle = null,
state = null,
source = source,
)
}.getOrNull()
}
}.awaitAll().filterNotNull()
}
override suspend fun getDetails(manga: Manga): Manga {
val json = webClient.httpGet("$ltnBaseUrl/galleries/${manga.url}.js")
.parseRaw()
.substringAfter("var galleryinfo = ")
.let(::JSONObject)
return manga.copy(
title = json.getString("title"),
largeCoverUrl =
json.getJSONArray("files").getJSONObject(0).let {
val hash = it.getString("hash")
val commonId = commonImageId()
val imageId = imageIdFromHash(hash)
val subDomain = 'a' + subdomainOffset(imageId)
"https://${getDomain("${subDomain}a")}/webp/$commonId$imageId/$hash.webp"
},
author =
json.optJSONArray("artists")
?.mapJSON { it.getString("artist").toCamelCase() }
?.joinToString(),
publicUrl = json.getString("galleryurl").toAbsoluteUrl(domain),
tags =
buildSet {
json.optJSONArray("characters")
?.mapToTags("character")
?.let(::addAll)
json.optJSONArray("tags")
?.mapToTags("tag")
?.let(::addAll)
json.optJSONArray("artists")
?.mapToTags("artist")
?.let(::addAll)
json.optJSONArray("parodys")
?.mapToTags("parody")
?.let(::addAll)
json.optJSONArray("groups")
?.mapToTags("group")
?.let(::addAll)
},
chapters = listOf(
MangaChapter(
id = generateUid(manga.url),
url = manga.url,
name = json.getString("title"),
scanlator = json.getString("type").toTitleCase(),
number = 1,
branch = json.getString("language_localname"),
source = source,
uploadDate = dateFormat.tryParse(json.getString("date").substringBeforeLast("-")),
),
),
)
}
private val dateFormat = SimpleDateFormat("yyyy-MM-dd HH:mm:ss", Locale.ENGLISH)
private fun JSONArray.mapToTags(key: String): Set<MangaTag> {
val tags = ArraySet<MangaTag>(length())
mapJSON {
MangaTag(
title =
it.getString(key).toCamelCase().let { title ->
if (it.getStringOrNull("female")?.toIntOrNull() == 1) {
"$title ♀"
} else if (it.getStringOrNull("male")?.toIntOrNull() == 1) {
"$title ♂"
} else {
title
}
},
key = it.getString("url").tagUrlToTag(),
source = source,
).let(tags::add)
}
return tags
}
private fun String.tagUrlToTag(): String {
val urlContent = this.split("/")
val ns = urlContent[1]
val tag =
urlContent[2]
.substringBeforeLast("-")
.urlDecode()
.replace(" ", "_")
return if (tag.split(":")[0] in listOf("female", "male")) {
tag
} else {
"$ns:$tag"
}
}
override suspend fun getRelatedManga(seed: Manga): List<Manga> {
val json = webClient.httpGet("$ltnBaseUrl/galleries/${seed.url}.js")
.parseRaw()
.substringAfter("var galleryinfo = ")
.let(::JSONObject)
// any better way to get List<Int> from this json?
return json.getJSONArray("related").let {
0.until(it.length()).map { i -> it.getInt(i) }
}.toMangaList()
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val json = webClient.httpGet("$ltnBaseUrl/galleries/${chapter.url}.js")
.parseRaw()
.substringAfter("var galleryinfo = ")
.let(::JSONObject)
return json.getJSONArray("files").mapJSON { image ->
val hash = image.getString("hash")
val commonId = commonImageId()
val imageId = imageIdFromHash(hash)
val subDomain = 'a' + subdomainOffset(imageId)
MangaPage(
id = generateUid(hash),
url = "https://${getDomain("${subDomain}a")}/webp/$commonId$imageId/$hash.webp",
preview = "https://${getDomain("${subDomain}tn")}/webpsmalltn/${thumbPathFromHash(hash)}/$hash.webp",
source = source,
)
}
}
// / --->
private var scriptLastRetrieval: Long? = null
private val mutex = Mutex()
private var subdomainOffsetDefault = 0
private val subdomainOffsetMap = mutableMapOf<Int, Int>()
private var commonImageId = ""
private suspend fun refreshScript() = mutex.withLock {
if (scriptLastRetrieval == null || (scriptLastRetrieval!! + 60000) < System.currentTimeMillis()) {
val ggScript = webClient.httpGet("$ltnBaseUrl/gg.js?_=${System.currentTimeMillis()}").parseRaw()
subdomainOffsetDefault = Regex("var o = (\\d)").find(ggScript)!!.groupValues[1].toInt()
val o = Regex("o = (\\d); break;").find(ggScript)!!.groupValues[1].toInt()
subdomainOffsetMap.clear()
Regex("case (\\d+):").findAll(ggScript).forEach {
val case = it.groupValues[1].toInt()
subdomainOffsetMap[case] = o
}
commonImageId = Regex("b: '(.+)'").find(ggScript)!!.groupValues[1]
scriptLastRetrieval = System.currentTimeMillis()
}
}
// m <-- gg.js
private suspend fun subdomainOffset(imageId: Int): Int {
refreshScript()
return subdomainOffsetMap[imageId] ?: subdomainOffsetDefault
}
// b <-- gg.js
private suspend fun commonImageId(): String {
refreshScript()
return commonImageId
}
// s <-- gg.js
private fun imageIdFromHash(hash: String): Int {
val match = Regex("(..)(.)$").find(hash)
return match!!.groupValues.let { it[2] + it[1] }.toInt(16)
}
// real_full_path_from_hash <-- common.js
private fun thumbPathFromHash(hash: String): String {
return hash.replace(Regex("""^.*(..)(.)$"""), "$2/$1")
}
private suspend fun subdomainFromURL(url: String, base: String? = null): String {
var retval = "b"
if (!base.isNullOrBlank())
retval = base
val regex = Regex("""/[0-9a-f]{61}([0-9a-f]{2})([0-9a-f])""")
val hashMatch = regex.find(url) ?: return "a"
val imageId = hashMatch.groupValues.let { it[2] + it[1] }.toIntOrNull(16)
if (imageId != null) {
retval = ('a' + subdomainOffset(imageId)).toString() + retval
}
return retval
}
// rewrite_tn_paths <-- common.js
private suspend fun rewriteTnPaths(html: String): String {
val tnRegex = Regex("""//tn\.hitomi\.la/[^/]+/[0-9a-f]/[0-9a-f]{2}/[0-9a-f]{64}""")
val url = tnRegex.find(html)?.value ?: return html
val newSubdomain = subdomainFromURL(url, "tn")
val newUrl = url.replace(Regex("""//..?\.hitomi\.la/"""), "//${getDomain(newSubdomain)}/")
return html.replace(tnRegex, newUrl)
}
private fun String.toTagTitle(): String {
return toCamelCase()
.replace("♂", "(male)")
.replace("♀", "(female)")
}
}

View File

@@ -0,0 +1,186 @@
package org.koitharu.kotatsu.parsers.site.all
import androidx.collection.ArraySet
import kotlinx.coroutines.async
import kotlinx.coroutines.awaitAll
import kotlinx.coroutines.coroutineScope
import org.jsoup.nodes.Element
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.util.*
@MangaSourceParser("IMHENTAI", "ImHentai", type = ContentType.HENTAI)
internal class ImHentai(context: MangaLoaderContext) :
PagedMangaParser(context, MangaSource.IMHENTAI, pageSize = 20) {
override val availableSortOrders: Set<SortOrder> =
EnumSet.of(SortOrder.UPDATED, SortOrder.POPULARITY, SortOrder.RATING)
override val configKeyDomain = ConfigKey.Domain("imhentai.xxx")
override suspend fun getListPage(page: Int, filter: MangaListFilter?): List<Manga> {
val url = buildString {
append("https://")
append(domain)
append("/search/?page=")
append(page.toString())
when (filter) {
is MangaListFilter.Search -> {
append("&key=")
append(filter.query.urlEncoded())
}
is MangaListFilter.Advanced -> {
if (filter.tags.isNotEmpty()) {
append("&key=")
append(filter.tags.joinToString(separator = ",") { it.key })
}
var lang = "&en=1&jp=1&es=1&fr=1&kr=1&de=1&ru=1"
filter.locale?.let {
lang = "&en=0&jp=0&es=0&fr=0&kr=0&de=0&ru=0"
lang = lang.replace("${it.language}=0", "${it.language}=1")
}
append(lang)
when (filter.sortOrder) {
SortOrder.UPDATED -> append("&lt=1&pp=0")
SortOrder.POPULARITY -> append("&lt=0&pp=1")
SortOrder.RATING -> append("&lt=0&pp=0")
else -> append("&lt=1&pp=0")
}
}
null -> {
append("&lt=1&pp=0")
}
}
}
val doc = webClient.httpGet(url).parseHtml()
return doc.select("div.galleries div.thumb").map { div ->
val a = div.selectFirstOrThrow(".inner_thumb a")
val href = a.attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(domain),
coverUrl = a.selectFirst("img")?.src().orEmpty(),
title = div.selectFirst(".caption")?.text().orEmpty(),
altTitle = null,
rating = RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = null,
source = source,
isNsfw = isNsfwSource,
)
}
}
//Tags are deliberately reduced because there are too many and this slows down the application.
//only the most popular ones are taken.
override suspend fun getAvailableTags(): Set<MangaTag> {
return coroutineScope {
(1..3).map { page ->
async { getTags(page) }
}
}.awaitAll().flattenTo(ArraySet(360))
}
private suspend fun getTags(page: Int): Set<MangaTag> {
val url = "https://$domain/tags/popular/?page=$page"
val root = webClient.httpGet(url).parseHtml()
return root.parseTags()
}
private fun Element.parseTags() = select("div.stags a.tag_btn").mapToSet {
val href = it.attr("href").substringAfterLast("tag/").substringBeforeLast('/')
MangaTag(
key = href,
title = it.selectFirstOrThrow("h3.list_tag").text().toTitleCase(sourceLocale),
source = source,
)
}
override suspend fun getAvailableLocales(): Set<Locale> = setOf(
Locale.ENGLISH, Locale.JAPANESE, Locale("es"), Locale.FRENCH, Locale("kr"), Locale.GERMAN, Locale("ru"),
)
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val fullUrl = manga.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
manga.copy(
tags = doc.body().select("li:contains(Tags) a.tag").mapNotNullToSet {
val href = it.attr("href").substringAfterLast("tag/").substringBeforeLast('/')
val name = it.html().substringBeforeLast("<span")
MangaTag(
key = href,
title = name,
source = source,
)
},
author = doc.selectFirst("li:contains(Artists) a.tag")?.html()?.substringBefore("<span"),
chapters = listOf(
MangaChapter(
id = manga.id,
name = manga.title,
number = 1,
url = manga.url,
scanlator = null,
uploadDate = 0,
branch = doc.selectFirst("li:contains(Language) a.tag")?.html()?.substringBeforeLast("<span"),
source = source,
),
),
)
}
override suspend fun getRelatedManga(seed: Manga): List<Manga> {
val doc = webClient.httpGet(seed.url.toAbsoluteUrl(domain)).parseHtml()
val root = doc.body().selectFirstOrThrow("div.related")
return root.select("div.thumb").mapNotNull { div ->
val a = div.selectFirstOrThrow(".inner_thumb a")
val href = a.attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(domain),
coverUrl = a.selectFirst("img")?.src().orEmpty(),
title = div.selectFirst(".caption")?.text().orEmpty(),
altTitle = null,
rating = RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = null,
source = source,
isNsfw = false,
)
}
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
val totalPages = doc.selectFirstOrThrow(".pages").text().replace("Pages: ", "").toInt() + 1
val domainImg = doc.requireElementById("append_thumbs").selectFirstOrThrow("img").src()?.replace("1t.jpg", "")
val pages = ArrayList<MangaPage>(totalPages)
for (i in 1 until totalPages) {
val url = "$domainImg$i.jpg"
pages.add(
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
),
)
}
return pages
}
}

View File

@@ -0,0 +1,354 @@
package org.koitharu.kotatsu.parsers.site.all
import kotlinx.coroutines.async
import kotlinx.coroutines.coroutineScope
import okhttp3.Headers
import okhttp3.HttpUrl
import okhttp3.HttpUrl.Companion.toHttpUrl
import org.json.JSONObject
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaParser
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.exception.NotFoundException
import org.koitharu.kotatsu.parsers.exception.ParseException
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import org.koitharu.kotatsu.parsers.util.json.*
import java.util.*
import javax.crypto.Mac
import javax.crypto.spec.SecretKeySpec
internal abstract class LineWebtoonsParser(
context: MangaLoaderContext,
source: MangaSource,
) : MangaParser(context, source) {
override val isMultipleTagsSupported = false
private val signer by lazy {
WebtoonsUrlSigner("gUtPzJFZch4ZyAGviiyH94P99lQ3pFdRTwpJWDlSGFfwgpr6ses5ALOxWHOIT7R1")
}
// we don't __really__ support changing this domain because:
// 1. I don't think other websites have this exact API
// 2. most communication is done with other domains (hosting API and static content), which are not configurable
// 3. we rely on the HTTP client setting the referer header to webtoons.com
//
// This effectively means that changing the domain will break the source. Yikes
override val configKeyDomain = ConfigKey.Domain("webtoons.com")
private val apiDomain = "global.apis.naver.com"
private val staticDomain = "webtoon-phinf.pstatic.net"
override val availableSortOrders: Set<SortOrder> = EnumSet.of(
SortOrder.POPULARITY,
// doesn't actually sort by rating, but by likes
// this should be fine though
SortOrder.RATING,
SortOrder.UPDATED,
)
override val headers: Headers
get() = Headers.Builder()
.add("User-Agent", "nApps (Android 12;; linewebtoon; 3.1.0)")
.build()
override suspend fun getPageUrl(page: MangaPage): String {
return page.url.toAbsoluteUrl(staticDomain)
}
// some language tags do not map perfectly to the ones used by the API
private val languageCode: String
get() = when (val tag = sourceLocale.toLanguageTag()) {
"in" -> "id"
"zh" -> "zh-hant"
else -> tag
}
private suspend fun getChapters(titleNo: Long): List<MangaChapter> {
val firstResult = makeRequest(
url = "/lineWebtoon/webtoon/challengeEpisodeList.json?v=2&titleNo=$titleNo&startIndex=0&pageSize=30",
)
val totalEpisodeCount = firstResult
.getJSONObject("episodeList")
.getInt("totalServiceEpisodeCount")
val episodes = firstResult
.getJSONObject("episodeList")
.getJSONArray("episode")
.toJSONList()
.toMutableList()
while (episodes.count() < totalEpisodeCount) {
val page = makeRequest(
url = "/lineWebtoon/webtoon/challengeEpisodeList.json?v=2&titleNo=$titleNo&startIndex=${episodes.count()}&pageSize=30",
).getJSONObject("episodeList")
.getJSONArray("episode")
.toJSONList()
episodes.addAll(page)
}
return episodes.mapChapters { i, jo ->
MangaChapter(
id = generateUid("$titleNo-$i"),
name = jo.getString("episodeTitle"),
number = jo.getInt("episodeSeq"),
url = "$titleNo-${jo.get("episodeNo")}",
uploadDate = jo.getLong("modifyYmdt"),
branch = null,
scanlator = null,
source = source,
)
}.sortedBy { it.number }
}
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val titleNo = manga.url.toLong()
val chaptersDeferred = async { getChapters(titleNo) }
makeRequest("/lineWebtoon/webtoon/challengeTitleInfo.json?v=2&titleNo=${titleNo}")
.getJSONObject("titleInfo")
.let { jo ->
Manga(
id = generateUid(titleNo),
title = jo.getString("title"),
altTitle = null,
url = "$titleNo",
publicUrl = "https://$domain/$languageCode/canvas/a/list?title_no=${titleNo}",
rating = jo.getFloatOrDefault("starScoreAverage", -10f) / 10f,
isNsfw = jo.getBooleanOrDefault("ageGradeNotice", isNsfwSource),
coverUrl = jo.getString("thumbnail").toAbsoluteUrl(staticDomain),
largeCoverUrl = jo.getStringOrNull("thumbnailVertical")?.toAbsoluteUrl(staticDomain),
tags = setOf(parseTag(jo.getJSONObject("genreInfo"))),
author = jo.getStringOrNull("writingAuthorName"),
description = jo.getString("synopsis"),
// I don't think the API provides this info
state = null,
chapters = chaptersDeferred.await(),
source = source,
)
}
}
override suspend fun getList(offset: Int, filter: MangaListFilter?): List<Manga> {
val manga =
when (filter) {
is MangaListFilter.Search -> {
makeRequest("/lineWebtoon/webtoon/searchChallenge?query=${filter.query.urlEncoded()}&startIndex=${offset + 1}&pageSize=20")
.getJSONObject("challengeSearch")
.getJSONArray("titleList")
.mapJSON { jo ->
val titleNo = jo.getLong("titleNo")
Manga(
id = generateUid(titleNo),
title = jo.getString("title"),
altTitle = null,
url = titleNo.toString(),
publicUrl = "https://$domain/$languageCode/canvas/a/list?title_no=$titleNo",
rating = RATING_UNKNOWN,
isNsfw = isNsfwSource,
coverUrl = jo.getString("thumbnail").toAbsoluteUrl(staticDomain),
largeCoverUrl = null,
tags = emptySet(),
author = jo.getStringOrNull("writingAuthorName"),
description = null,
state = null,
source = source,
)
}
}
is MangaListFilter.Advanced -> {
val genre = filter.tags.oneOrThrowIfMany()?.key ?: "ALL"
val sortOrderStr = when (filter.sortOrder) {
SortOrder.UPDATED -> "UPDATE"
SortOrder.POPULARITY -> "READ_COUNT"
SortOrder.RATING -> "LIKEIT"
else -> throw IllegalArgumentException("Unsupported sort order: ${filter.sortOrder}")
}
val result =
makeRequest("/lineWebtoon/webtoon/challengeGenreTitleList.json?genre=$genre&sortOrder=$sortOrderStr&startIndex=${offset + 1}&pageSize=20")
val genres = result.getJSONObject("genreList")
.getJSONArray("challengeGenres")
.mapJSON { jo -> parseTag(jo) }
.associateBy { tag -> tag.key }
result
.getJSONObject("titleList")
.getJSONArray("titles")
.mapJSON { jo ->
val titleNo = jo.getLong("titleNo")
Manga(
id = generateUid(titleNo),
title = jo.getString("title"),
altTitle = null,
url = titleNo.toString(),
publicUrl = "https://$domain/$languageCode/canvas/a/list?title_no=$titleNo",
rating = jo.getFloatOrDefault("starScoreAverage", -10f) / 10f,
isNsfw = jo.getBooleanOrDefault("ageGradeNotice", isNsfwSource),
coverUrl = jo.getString("thumbnail").toAbsoluteUrl(staticDomain),
largeCoverUrl = jo.getStringOrNull("thumbnailVertical")?.toAbsoluteUrl(staticDomain),
tags = setOfNotNull(genres[jo.getString("representGenre")]),
author = jo.getStringOrNull("writingAuthorName"),
description = jo.getString("synopsis"),
// I don't think the API provides this info
state = null,
source = source,
)
}
}
null -> {
val result =
makeRequest("/lineWebtoon/webtoon/challengeGenreTitleList.json?genre=ALL&sortOrder=UPDATE&startIndex=${offset + 1}&pageSize=20")
val genres = result.getJSONObject("genreList")
.getJSONArray("challengeGenres")
.mapJSON { jo -> parseTag(jo) }
.associateBy { tag -> tag.key }
result
.getJSONObject("titleList")
.getJSONArray("titles")
.mapJSON { jo ->
val titleNo = jo.getLong("titleNo")
Manga(
id = generateUid(titleNo),
title = jo.getString("title"),
altTitle = null,
url = titleNo.toString(),
publicUrl = "https://$domain/$languageCode/canvas/a/list?title_no=$titleNo",
rating = jo.getFloatOrDefault("starScoreAverage", -10f) / 10f,
isNsfw = jo.getBooleanOrDefault("ageGradeNotice", isNsfwSource),
coverUrl = jo.getString("thumbnail").toAbsoluteUrl(staticDomain),
largeCoverUrl = jo.getStringOrNull("thumbnailVertical")?.toAbsoluteUrl(staticDomain),
tags = setOfNotNull(genres[jo.getString("representGenre")]),
author = jo.getStringOrNull("writingAuthorName"),
description = jo.getString("synopsis"),
// I don't think the API provides this info
state = null,
source = source,
)
}
}
}
return manga
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val (titleNo, episodeNo) = requireNotNull(chapter.url.splitTwoParts('-'))
return makeRequest("/lineWebtoon/webtoon/challengeEpisodeInfo.json?v=2&titleNo=$titleNo&episodeNo=$episodeNo")
.getJSONObject("episodeInfo")
.getJSONArray("imageInfo")
.mapJSONIndexed { i, jo ->
MangaPage(
id = generateUid("$titleNo-$episodeNo-$i"),
url = jo.getString("url"),
preview = null,
source = source,
)
}
}
private fun parseTag(jo: JSONObject): MangaTag {
return MangaTag(
title = jo.getString("name"),
key = jo.getString("code"),
source = source,
)
}
override suspend fun getAvailableTags(): Set<MangaTag> {
return makeRequest("/lineWebtoon/webtoon/challengeGenreList.json")
.getJSONObject("genreList")
.getJSONArray("challengeGenres")
.mapJSONToSet { jo -> parseTag(jo) }
}
private suspend fun makeRequest(url: String): JSONObject {
val resp = webClient.httpGet(finalizeUrl(url))
val message: JSONObject? = resp.parseJson().optJSONObject("message")
return when (resp.code) {
in 200..299 -> checkNotNull(message).getJSONObject("result")
404 -> throw NotFoundException(message?.getStringOrNull("message").orEmpty(), url)
else -> {
val code = message?.getIntOrDefault("code", 0)
val errorMessage = message?.getStringOrNull("message")
throw ParseException("Api error (code=$code): $errorMessage", url)
}
}
}
private fun finalizeUrl(url: String): HttpUrl {
val httpUrl = url.toAbsoluteUrl(apiDomain).toHttpUrl()
val builder = httpUrl.newBuilder()
.addQueryParameter("serviceZone", "GLOBAL")
if (httpUrl.queryParameter("v") == null) {
builder.addQueryParameter("v", "1")
}
builder.addQueryParameter("language", languageCode)
.addQueryParameter("locale", "languageCode")
.addQueryParameter("platform", "APP_ANDROID")
signer.makeEncryptUrl(builder)
return builder.build()
}
@MangaSourceParser("LINEWEBTOONS_EN", "LineWebtoons English", "en", type = ContentType.MANGA)
class English(context: MangaLoaderContext) : LineWebtoonsParser(context, MangaSource.LINEWEBTOONS_EN)
@MangaSourceParser("LINEWEBTOONS_ZH", "LineWebtoons Chinese", "zh", type = ContentType.MANGA)
class Chinese(context: MangaLoaderContext) : LineWebtoonsParser(context, MangaSource.LINEWEBTOONS_ZH)
@MangaSourceParser("LINEWEBTOONS_TH", "LineWebtoons Thai", "th", type = ContentType.MANGA)
class Thai(context: MangaLoaderContext) : LineWebtoonsParser(context, MangaSource.LINEWEBTOONS_TH)
@MangaSourceParser("LINEWEBTOONS_ID", "LineWebtoons Indonesian", "id", type = ContentType.MANGA)
class Indonesian(context: MangaLoaderContext) : LineWebtoonsParser(context, MangaSource.LINEWEBTOONS_ID)
@MangaSourceParser("LINEWEBTOONS_ES", "LineWebtoons Spanish", "es", type = ContentType.MANGA)
class Spanish(context: MangaLoaderContext) : LineWebtoonsParser(context, MangaSource.LINEWEBTOONS_ES)
@MangaSourceParser("LINEWEBTOONS_FR", "LineWebtoons French", "fr", type = ContentType.MANGA)
class French(context: MangaLoaderContext) : LineWebtoonsParser(context, MangaSource.LINEWEBTOONS_FR)
@MangaSourceParser("LINEWEBTOONS_DE", "LineWebtoons German", "de", type = ContentType.MANGA)
class German(context: MangaLoaderContext) : LineWebtoonsParser(context, MangaSource.LINEWEBTOONS_DE)
private inner class WebtoonsUrlSigner(private val secret: String) {
private val mac = Mac.getInstance("HmacSHA1").apply {
this.init(SecretKeySpec(secret.encodeToByteArray(), "HmacSHA1"))
}
private fun getMessage(url: String, msgpad: String): String {
return url.substring(0, 0xFF.coerceAtMost(url.length)) + msgpad
}
private fun getMessageDigest(s: String): String {
val signedMessage = synchronized(mac) { mac.doFinal(s.toByteArray()) }
return context.encodeBase64(signedMessage)
}
fun makeEncryptUrl(urlBuilder: HttpUrl.Builder) {
val msgPad = Calendar.getInstance().timeInMillis.toString()
val digest = getMessageDigest(getMessage(urlBuilder.build().toString(), msgPad))
urlBuilder
.addQueryParameter("msgpad", msgPad)
.addQueryParameter("md", digest)
// .addEncodedQueryParameter("md", digest.urlEncoded())
}
}
}

View File

@@ -1,4 +1,4 @@
package org.koitharu.kotatsu.parsers.site
package org.koitharu.kotatsu.parsers.site.all
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.async
@@ -20,29 +20,26 @@ private const val PAGE_SIZE = 20
private const val CHAPTERS_FIRST_PAGE_SIZE = 120
private const val CHAPTERS_MAX_PAGE_SIZE = 500
private const val CHAPTERS_PARALLELISM = 3
private const val CONTENT_RATING =
"contentRating[]=safe&contentRating[]=suggestive&contentRating[]=erotica&contentRating[]=pornographic"
private const val CHAPTERS_MAX_COUNT = 10_000 // strange api behavior, looks like a bug
private const val LOCALE_FALLBACK = "en"
@MangaSourceParser("MANGADEX", "MangaDex")
internal class MangaDexParser(override val context: MangaLoaderContext) : MangaParser(MangaSource.MANGADEX) {
internal class MangaDexParser(context: MangaLoaderContext) : MangaParser(context, MangaSource.MANGADEX) {
override val configKeyDomain = ConfigKey.Domain("mangadex.org", null)
override val configKeyDomain = ConfigKey.Domain("mangadex.org")
override val sortOrders: EnumSet<SortOrder> = EnumSet.of(
SortOrder.UPDATED,
SortOrder.ALPHABETICAL,
SortOrder.NEWEST,
SortOrder.POPULARITY,
)
override val availableSortOrders: EnumSet<SortOrder> = EnumSet.allOf(SortOrder::class.java)
override suspend fun getList(
offset: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val domain = getDomain()
override val availableContentRating: Set<ContentRating> = EnumSet.allOf(ContentRating::class.java)
override val availableStates: Set<MangaState> =
EnumSet.of(MangaState.ONGOING, MangaState.FINISHED, MangaState.PAUSED, MangaState.ABANDONED)
override val isTagsExclusionSupported: Boolean = true
override suspend fun getList(offset: Int, filter: MangaListFilter?): List<Manga> {
val domain = domain
val url = buildString {
append("https://api.")
append(domain)
@@ -50,32 +47,67 @@ internal class MangaDexParser(override val context: MangaLoaderContext) : MangaP
append(PAGE_SIZE)
append("&offset=")
append(offset)
append("&includes[]=cover_art&includes[]=author&includes[]=artist&")
tags?.forEach { tag ->
append("includedTags[]=")
append(tag.key)
append('&')
}
if (!query.isNullOrEmpty()) {
append("title=")
append(query.urlEncoded())
append('&')
}
append(CONTENT_RATING)
append("&order")
append(
when (sortOrder) {
SortOrder.UPDATED,
-> "[latestUploadedChapter]=desc"
append("&includes[]=cover_art&includes[]=author&includes[]=artist")
when (filter) {
is MangaListFilter.Search -> {
append("&title=")
append(filter.query)
}
SortOrder.ALPHABETICAL -> "[title]=asc"
SortOrder.NEWEST -> "[createdAt]=desc"
SortOrder.POPULARITY -> "[followedCount]=desc"
else -> "[followedCount]=desc"
},
)
is MangaListFilter.Advanced -> {
filter.tags.forEach {
append("&includedTags[]=")
append(it.key)
}
filter.tagsExclude.forEach {
append("&excludedTags[]=")
append(it.key)
}
if (filter.contentRating.isNotEmpty()) {
filter.contentRating.forEach {
when (it) {
ContentRating.SAFE -> append("&contentRating[]=safe")
ContentRating.SUGGESTIVE -> append("&contentRating[]=suggestive&contentRating[]=erotica")
ContentRating.ADULT -> append("&contentRating[]=pornographic")
}
}
}
append("&order")
append(
when (filter.sortOrder) {
SortOrder.UPDATED -> "[latestUploadedChapter]=desc"
SortOrder.RATING -> "[rating]=desc"
SortOrder.ALPHABETICAL -> "[title]=asc"
SortOrder.ALPHABETICAL_DESC -> "[title]=desc"
SortOrder.NEWEST -> "[createdAt]=desc"
SortOrder.POPULARITY -> "[followedCount]=desc"
},
)
filter.states.forEach {
append("&status[]=")
when (it) {
MangaState.ONGOING -> append("ongoing")
MangaState.FINISHED -> append("completed")
MangaState.ABANDONED -> append("cancelled")
MangaState.PAUSED -> append("hiatus")
else -> append("")
}
}
filter.locale?.let {
append("&availableTranslatedLanguage[]=")
append(it.language)
}
}
null -> {
append("&order[latestUploadedChapter]=desc")
}
}
}
val json = context.httpGet(url).parseJson().getJSONArray("data")
val json = webClient.httpGet(url).parseJson().getJSONArray("data")
return json.mapJSON { jo ->
val id = jo.getString("id")
val attrs = jo.getJSONObject("attributes")
@@ -95,7 +127,10 @@ internal class MangaDexParser(override val context: MangaLoaderContext) : MangaP
url = id,
publicUrl = "https://$domain/title/$id",
rating = RATING_UNKNOWN,
isNsfw = attrs.getStringOrNull("contentRating") == "erotica",
isNsfw = when (attrs.getStringOrNull("contentRating")) {
"erotica", "pornographic" -> true
else -> false
},
coverUrl = cover?.plus(".256.jpg").orEmpty(),
largeCoverUrl = cover,
description = attrs.optJSONObject("description")?.selectByLocale(),
@@ -109,9 +144,11 @@ internal class MangaDexParser(override val context: MangaLoaderContext) : MangaP
source = source,
)
},
state = when (jo.getStringOrNull("status")) {
state = when (attrs.getStringOrNull("status")) {
"ongoing" -> MangaState.ONGOING
"completed" -> MangaState.FINISHED
"hiatus" -> MangaState.PAUSED
"cancelled" -> MangaState.ABANDONED
else -> null
},
author = (relations["author"] ?: relations["artist"])
@@ -123,80 +160,64 @@ internal class MangaDexParser(override val context: MangaLoaderContext) : MangaP
}
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val domain = getDomain()
val domain = domain
val mangaId = manga.url.removePrefix("/")
val attrsDeferred = async {
context.httpGet(
webClient.httpGet(
"https://api.$domain/manga/${mangaId}?includes[]=artist&includes[]=author&includes[]=cover_art",
).parseJson().getJSONObject("data").getJSONObject("attributes")
}
val feedDeferred = async { loadChapters(mangaId) }
val mangaAttrs = attrsDeferred.await()
val feed = feedDeferred.await()
// 2022-01-02T00:27:11+00:00
val dateFormat = SimpleDateFormat(
"yyyy-MM-dd'T'HH:mm:ss'+00:00'",
Locale.ROOT,
)
manga.copy(
description = mangaAttrs.getJSONObject("description").selectByLocale()
description = mangaAttrs.optJSONObject("description")?.selectByLocale()
?: manga.description,
chapters = feed.mapChapters { _, jo ->
val id = jo.getString("id")
val attrs = jo.getJSONObject("attributes")
if (!attrs.isNull("externalUrl")) {
return@mapChapters null
}
val locale = attrs.getStringOrNull("translatedLanguage")?.let { Locale.forLanguageTag(it) }
val relations = jo.getJSONArray("relationships").associateByKey("type")
val number = attrs.getIntOrDefault("chapter", 0)
MangaChapter(
id = generateUid(id),
name = attrs.getStringOrNull("title")?.takeUnless(String::isEmpty)
?: "Chapter #$number",
number = number,
url = id,
scanlator = relations["scanlation_group"]?.getStringOrNull("name"),
uploadDate = dateFormat.tryParse(attrs.getString("publishAt")),
branch = locale?.getDisplayName(locale)?.toTitleCase(locale),
source = source,
)
},
chapters = mapChapters(feed),
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val domain = getDomain()
val chapterJson = context.httpGet("https://api.$domain/at-home/server/${chapter.url}?forcePort443=false")
val domain = domain
val chapterJson = webClient.httpGet("https://api.$domain/at-home/server/${chapter.url}?forcePort443=false")
.parseJson()
.getJSONObject("chapter")
val pages = chapterJson.getJSONArray("data")
val prefix = "https://uploads.$domain/data/${chapterJson.getString("hash")}/"
val referer = "https://$domain/"
return List(pages.length()) { i ->
val url = prefix + pages.getString(i)
MangaPage(
id = generateUid(url),
url = url,
referer = referer,
preview = null, // TODO prefix + dataSaver.getString(i),
source = source,
)
}
}
override suspend fun getTags(): Set<MangaTag> {
val tags = context.httpGet("https://api.${getDomain()}/manga/tag").parseJson()
override suspend fun getAvailableTags(): Set<MangaTag> {
val tags = webClient.httpGet("https://api.${domain}/manga/tag").parseJson()
.getJSONArray("data")
return tags.mapJSONToSet { jo ->
MangaTag(
title = jo.getJSONObject("attributes").getJSONObject("name").firstStringValue().toTitleCase(),
title = jo.getJSONObject("attributes").getJSONObject("name")
.firstStringValue()
.toTitleCase(Locale.ENGLISH),
key = jo.getString("id"),
source = source,
)
}
}
override suspend fun getAvailableLocales(): Set<Locale> {
val head = webClient.httpGet("https://$domain/").parseHtml().head()
return head.getElementsByAttributeValue("property", "og:locale:alternate")
.mapNotNullToSet { meta ->
val raw = meta.attrOrNull("content") ?: return@mapNotNullToSet null
Locale(raw.substringBefore('_'), raw.substringAfter('_', ""))
}
}
private fun JSONObject.firstStringValue() = values().next() as String
private fun JSONObject.selectByLocale(): String? {
@@ -214,7 +235,7 @@ internal class MangaDexParser(override val context: MangaLoaderContext) : MangaP
return firstPage.data
}
val tail = coroutineScope {
val leftCount = firstPage.total - firstPage.size
val leftCount = firstPage.total.coerceAtMost(CHAPTERS_MAX_COUNT) - firstPage.size
val pages = (leftCount / CHAPTERS_MAX_PAGE_SIZE.toFloat()).toIntUp()
val dispatcher = Dispatchers.Default.limitedParallelism(CHAPTERS_PARALLELISM)
List(pages) { page ->
@@ -231,20 +252,24 @@ internal class MangaDexParser(override val context: MangaLoaderContext) : MangaP
}
private suspend fun loadChapters(mangaId: String, offset: Int, limit: Int): Chapters {
val limitedLimit = when {
offset >= CHAPTERS_MAX_COUNT -> return Chapters(emptyList(), CHAPTERS_MAX_COUNT)
offset + limit > CHAPTERS_MAX_COUNT -> CHAPTERS_MAX_COUNT - offset
else -> limit
}
val url = buildString {
append("https://api.")
append(getDomain())
append(domain)
append("/manga/")
append(mangaId)
append("/feed")
append("?limit=")
append(limit)
append(limitedLimit)
append("&includes[]=scanlation_group&order[volume]=asc&order[chapter]=asc&offset=")
append(offset)
append('&')
append(CONTENT_RATING)
append("&contentRating[]=safe&contentRating[]=suggestive&contentRating[]=erotica&contentRating[]=pornographic")
}
val json = context.httpGet(url).parseJson()
val json = webClient.httpGet(url).parseJson()
if (json.getString("result") == "ok") {
return Chapters(
data = json.optJSONArray("data")?.toJSONList().orEmpty(),
@@ -258,6 +283,48 @@ internal class MangaDexParser(override val context: MangaLoaderContext) : MangaP
}
}
private fun mapChapters(list: List<JSONObject>): List<MangaChapter> {
// 2022-01-02T00:27:11+00:00
val dateFormat = SimpleDateFormat(
"yyyy-MM-dd'T'HH:mm:ss'+00:00'",
Locale.ROOT,
)
val chaptersBuilder = ChaptersListBuilder(list.size)
val branchedChapters = HashMap<String?, HashMap<Float, MangaChapter>>()
for (jo in list) {
val id = jo.getString("id")
val attrs = jo.getJSONObject("attributes")
if (!attrs.isNull("externalUrl")) {
continue
}
val number = jo.getJSONObject("attributes").getFloatOrDefault("chapter", 0f)
val locale = attrs.getStringOrNull("translatedLanguage")?.let { Locale.forLanguageTag(it) }
val lc = locale?.getDisplayName(locale)?.toTitleCase(locale)
val relations = jo.getJSONArray("relationships").associateByKey("type")
val team = relations["scanlation_group"]?.getJSONObject("attributes")?.getStringOrNull("name")
?.takeUnless { it.isBlank() }
val branch = (list.indices).firstNotNullOf { i ->
val b = if (i == 0) lc else "$lc ($i)"
if (branchedChapters[b]?.get(number) == null) b else null
}
val chapter = MangaChapter(
id = generateUid(id),
name = attrs.getStringOrNull("title")?.takeUnless(String::isEmpty)
?: "Chapter #${number.toString().removeSuffix(".0")}",
number = if (number <= 0f) (branchedChapters[branch]?.size?.plus(1) ?: 0) else number.toInt(),
url = id,
scanlator = team,
uploadDate = dateFormat.tryParse(attrs.getString("publishAt")),
branch = branch,
source = source,
)
if (chaptersBuilder.add(chapter)) {
branchedChapters.getOrPut(branch, ::HashMap)[number] = chapter
}
}
return chaptersBuilder.toList()
}
private class Chapters(
val data: List<JSONObject>,
val total: Int,
@@ -266,4 +333,4 @@ internal class MangaDexParser(override val context: MangaLoaderContext) : MangaP
val size: Int
get() = data.size
}
}
}

View File

@@ -0,0 +1,268 @@
package org.koitharu.kotatsu.parsers.site.all
import androidx.collection.ArrayMap
import kotlinx.coroutines.coroutineScope
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.text.DateFormat
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("MANGAPARK", "MangaPark")
internal class MangaPark(context: MangaLoaderContext) :
PagedMangaParser(context, MangaSource.MANGAPARK, pageSize = 36) {
override val availableSortOrders: Set<SortOrder> = EnumSet.allOf(SortOrder::class.java)
override val availableStates: Set<MangaState> = EnumSet.allOf(MangaState::class.java)
override val availableContentRating: Set<ContentRating> = EnumSet.of(ContentRating.SAFE)
override val isTagsExclusionSupported: Boolean = true
override val configKeyDomain = ConfigKey.Domain("mangapark.net")
private val tagsMap = SuspendLazy(::parseTags)
init {
context.cookieJar.insertCookies(domain, "nsfw", "2")
}
override suspend fun getListPage(page: Int, filter: MangaListFilter?): List<Manga> {
val url = buildString {
append("https://")
append(domain)
append("/search?page=")
append(page.toString())
when (filter) {
is MangaListFilter.Search -> {
append("&word=")
append(filter.query.urlEncoded())
}
is MangaListFilter.Advanced -> {
append("&genres=")
if (filter.tags.isNotEmpty()) {
appendAll(filter.tags, ",") { it.key }
}
append("|")
if (filter.tagsExclude.isNotEmpty()) {
appendAll(filter.tagsExclude, ",") { it.key }
}
if (filter.contentRating.isNotEmpty()) {
filter.contentRating.oneOrThrowIfMany()?.let {
append(
when (it) {
ContentRating.SAFE -> append(",gore,bloody,violence,ecchi,adult,mature,smut,hentai")
else -> append("")
},
)
}
}
filter.states.oneOrThrowIfMany()?.let {
append("&status=")
append(
when (it) {
MangaState.ONGOING -> "ongoing"
MangaState.FINISHED -> "completed"
MangaState.PAUSED -> "hiatus"
MangaState.ABANDONED -> "cancelled"
MangaState.UPCOMING -> "pending"
},
)
}
append("&sortby=")
append(
when (filter.sortOrder) {
SortOrder.POPULARITY -> "views_d000"
SortOrder.UPDATED -> "field_update"
SortOrder.NEWEST -> "field_create"
SortOrder.ALPHABETICAL -> "field_name"
SortOrder.RATING -> "field_score"
else -> ""
},
)
filter.locale?.let {
append("&lang=")
append(it.language)
}
}
null -> append("&sortby=field_update")
}
}
val doc = webClient.httpGet(url).parseHtml()
return doc.select("div.grid.gap-5 div.flex.border-b").map { div ->
val href = div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(div.host ?: domain),
coverUrl = div.selectFirst("img")?.src().orEmpty(),
title = div.selectFirst("h3")?.text().orEmpty(),
altTitle = null,
rating = div.selectFirst("span.text-yellow-500")?.text()?.toFloatOrNull()?.div(10F) ?: RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = null,
source = source,
isNsfw = isNsfwSource,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
return tagsMap.get().values.toSet()
}
private suspend fun parseTags(): Map<String, MangaTag> {
val tagElements = webClient.httpGet("https://$domain/search").parseHtml()
.select("div.flex-col:contains(Genres) div.whitespace-nowrap")
val tagMap = ArrayMap<String, MangaTag>(tagElements.size)
for (el in tagElements) {
val name = el.selectFirstOrThrow("span.whitespace-nowrap").text().toTitleCase(sourceLocale)
if (name.isEmpty()) continue
tagMap[name] = MangaTag(
title = name,
key = el.attr("q:key") ?: continue,
source = source,
)
}
return tagMap
}
override suspend fun getAvailableLocales(): Set<Locale> = setOf(
Locale("af"), Locale("sq"), Locale("am"), Locale("ar"), Locale("hy"),
Locale("az"), Locale("be"), Locale("bn"), Locale("zh_hk"), Locale("zh_tw"),
Locale.CHINESE, Locale("ceb"), Locale("ca"), Locale("km"), Locale("my"),
Locale("bg"), Locale("bs"), Locale("hr"), Locale("cs"), Locale("da"),
Locale("nl"), Locale.ENGLISH, Locale("et"), Locale("fo"), Locale("fil"),
Locale("fi"), Locale("he"), Locale("ha"), Locale("jv"), Locale("lb"),
Locale("mn"), Locale("ro"), Locale("si"), Locale("ta"), Locale("uz"),
Locale("ur"), Locale("tg"), Locale("sd"), Locale("pt_br"), Locale("mo"),
Locale("lt"), Locale.JAPANESE, Locale.ITALIAN, Locale("ht"), Locale("lv"),
Locale("mr"), Locale("pt"), Locale("sn"), Locale("sv"), Locale("uk"),
Locale("tk"), Locale("sw"), Locale("st"), Locale("pl"), Locale("mi"),
Locale("lo"), Locale("ga"), Locale("gu"), Locale("gn"), Locale("id"),
Locale("ky"), Locale("mt"), Locale("fa"), Locale("sh"), Locale("es_419"),
Locale("tr"), Locale("to"), Locale("vi"), Locale("es"), Locale("sr"),
Locale("ps"), Locale("ml"), Locale("ku"), Locale("ig"), Locale("el"),
Locale.GERMAN, Locale("is"), Locale.KOREAN, Locale("ms"), Locale("ny"), Locale("sm"),
Locale("so"), Locale("ti"), Locale("zu"), Locale("yo"), Locale("th"),
Locale("sl"), Locale("ru"), Locale("no"), Locale("mg"), Locale("kk"),
Locale("hu"), Locale("ka"), Locale.FRENCH, Locale("hi"), Locale("kn"),
Locale("mk"), Locale("ne"), Locale("rm"), Locale("sk"), Locale("te"),
)
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val doc = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
val tagMap = tagsMap.get()
val selectTag = doc.select("div[q:key=30_2] span.whitespace-nowrap")
val tags = selectTag.mapNotNullToSet { tagMap[it.text()] }
val nsfw = tags.any { t -> t.key == "hentai" || t.key == "adult" }
val dateFormat = SimpleDateFormat("dd/MM/yyyy", sourceLocale)
manga.copy(
altTitle = doc.selectFirst("div[q:key=tz_2]")?.text().orEmpty(),
author = doc.selectFirst("div[q:key=tz_4]")?.text().orEmpty(),
description = doc.selectFirst("react-island[q:key=0a_9]")?.html().orEmpty(),
state = when (doc.selectFirst("span[q:key=Yn_5]")?.text()?.lowercase()) {
"ongoing" -> MangaState.ONGOING
"completed" -> MangaState.FINISHED
"hiatus" -> MangaState.PAUSED
"cancelled" -> MangaState.ABANDONED
else -> null
},
tags = tags,
isNsfw = nsfw,
chapters = doc.body().select("div.group.flex div.px-2").mapChapters { i, div ->
val a = div.selectFirstOrThrow("a")
val href = a.attrAsRelativeUrl("href")
val dateText = div.selectFirst("span[q:key=Ee_0]")?.text()
MangaChapter(
id = generateUid(href),
name = a.text(),
number = i + 1,
url = href,
uploadDate = parseChapterDate(
dateFormat,
dateText,
),
source = source,
scanlator = null,
branch = null,
)
},
)
}
private fun parseChapterDate(dateFormat: DateFormat, date: String?): Long {
val d = date?.lowercase() ?: return 0
return when {
d.endsWith(" ago") -> parseRelativeDate(date)
d.startsWith("just now") -> Calendar.getInstance().apply {
set(Calendar.HOUR_OF_DAY, 0)
set(Calendar.MINUTE, 0)
set(Calendar.SECOND, 0)
set(Calendar.MILLISECOND, 0)
}.timeInMillis
else -> dateFormat.tryParse(date)
}
}
private fun parseRelativeDate(date: String): Long {
val number = Regex("""(\d+)""").find(date)?.value?.toIntOrNull() ?: return 0
val cal = Calendar.getInstance()
return when {
WordSet("second").anyWordIn(date) -> cal.apply { add(Calendar.SECOND, -number) }.timeInMillis
WordSet("minute", "minutes", "mins", "min").anyWordIn(date) -> cal.apply {
add(
Calendar.MINUTE,
-number,
)
}.timeInMillis
WordSet("hour", "hours").anyWordIn(date) -> cal.apply { add(Calendar.HOUR, -number) }.timeInMillis
WordSet("day", "days").anyWordIn(date) -> cal.apply { add(Calendar.DAY_OF_MONTH, -number) }.timeInMillis
WordSet("month", "months").anyWordIn(date) -> cal.apply { add(Calendar.MONTH, -number) }.timeInMillis
WordSet("year").anyWordIn(date) -> cal.apply { add(Calendar.YEAR, -number) }.timeInMillis
else -> 0
}
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val doc = webClient.httpGet(chapter.url.toAbsoluteUrl(domain)).parseHtml()
val script = if (doc.selectFirst("script:containsData(comic-)") != null) {
doc.selectFirstOrThrow("script:containsData(comic-)").data()
.substringAfterLast("\"comic-").split("\",\"")
} else {
doc.selectFirstOrThrow("script:containsData(manga-)").data()
.substringAfterLast("\"manga-").split("\",\"")
}
return script.mapNotNull { url ->
if (!url.startsWith("https://")) {
return@mapNotNull null
} else {
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
)
}
}
}
}

View File

@@ -0,0 +1,321 @@
package org.koitharu.kotatsu.parsers.site.all
import okhttp3.Headers
import okhttp3.HttpUrl.Companion.toHttpUrl
import okhttp3.Interceptor
import okhttp3.MediaType.Companion.toMediaTypeOrNull
import okhttp3.Response
import okhttp3.ResponseBody.Companion.toResponseBody
import org.json.JSONArray
import org.json.JSONObject
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaParser
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import org.koitharu.kotatsu.parsers.util.json.getStringOrNull
import org.koitharu.kotatsu.parsers.util.json.mapJSON
import org.koitharu.kotatsu.parsers.util.json.mapJSONNotNull
import org.koitharu.kotatsu.parsers.util.json.toJSONList
import java.util.*
internal abstract class MangaPlusParser(
context: MangaLoaderContext,
source: MangaSource,
private val sourceLang: String,
) : MangaParser(context, source), Interceptor {
private val apiUrl = "https://jumpg-webapi.tokyo-cdn.com/api"
override val configKeyDomain = ConfigKey.Domain("mangaplus.shueisha.co.jp")
override val availableSortOrders: Set<SortOrder> = EnumSet.of(
SortOrder.POPULARITY,
SortOrder.UPDATED,
SortOrder.ALPHABETICAL,
)
private val extraHeaders = Headers.headersOf("Session-Token", UUID.randomUUID().toString())
// no tags or tag search available
override suspend fun getAvailableTags(): Set<MangaTag> = emptySet()
override suspend fun getList(offset: Int, filter: MangaListFilter?): List<Manga> {
if (offset > 0) {
return emptyList()
}
return when (filter) {
is MangaListFilter.Advanced -> {
when (filter.sortOrder) {
SortOrder.POPULARITY -> getPopularList()
SortOrder.UPDATED -> getLatestList()
else -> getAllTitleList()
}
}
is MangaListFilter.Search -> getAllTitleList(filter.query)
else -> getAllTitleList()
}
}
private suspend fun getPopularList(): List<Manga> {
val json = apiCall("/title_list/ranking")
return json.getJSONObject("titleRankingView")
.getJSONArray("titles")
.toJSONList()
.toMangaList()
}
private suspend fun getLatestList(): List<Manga> {
val json = apiCall("/title_list/updated")
return json.getJSONObject("titleUpdatedView")
.getJSONArray("latestTitle")
.mapJSON { it.getJSONObject("title") }
.toMangaList()
}
// since search is local, save network calls on related manga call
private val allTitleCache = SuspendLazy {
apiCall("/title_list/allV2")
.getJSONObject("allTitlesViewV2")
.getJSONArray("AllTitlesGroup")
.mapJSON { it.getJSONArray("titles").toJSONList() }
.flatten()
}
private suspend fun getAllTitleList(query: String? = null): List<Manga> {
return allTitleCache.get().toMangaList(query)
}
private fun List<JSONObject>.toMangaList(query: String? = null): List<Manga> {
return mapNotNull {
val language = it.getStringOrNull("language") ?: "ENGLISH"
if (language != sourceLang) {
return@mapNotNull null
}
val name = it.getString("name")
val author = it.getString("author")
.split('/')
.joinToString(transform = String::trim)
// filter out any other title or author which doesn't match search input
if (query != null && !(name.contains(query, true) || author.contains(query, true))) {
return@mapNotNull null
}
val titleId = it.getInt("titleId").toString()
Manga(
id = generateUid(titleId),
url = titleId,
publicUrl = "/titles/$titleId".toAbsoluteUrl(domain),
title = name,
coverUrl = it.getString("portraitImageUrl"),
altTitle = null,
author = author,
isNsfw = false,
rating = RATING_UNKNOWN,
state = null,
source = source,
tags = emptySet(),
)
}
}
override suspend fun getDetails(manga: Manga): Manga {
val json = apiCall("/title_detailV3?title_id=${manga.url}")
.getJSONObject("titleDetailView")
val title = json.getJSONObject("title")
val completed = json.getJSONObject("titleLabels")
.getString("releaseSchedule").let {
it == "DISABLED" || it == "COMPLETED"
}
val hiatus = json.getStringOrNull("nonAppearanceInfo")?.contains("on a hiatus") == true
return manga.copy(
title = title.getString("name"),
publicUrl = "/titles/${title.getInt("titleId")}".toAbsoluteUrl(domain),
coverUrl = title.getString("portraitImageUrl"),
author = title.getString("author")
.split("/").joinToString(transform = String::trim),
description = buildString {
json.getString("overview").let(::append)
json.getStringOrNull("viewingPeriodDescription")
?.takeIf { !completed }
?.let { append("<br><br>", it) }
},
chapters = parseChapters(
json.getJSONArray("chapterListGroup"),
title.getStringOrNull("language") ?: "ENGLISH",
),
state = when {
completed -> MangaState.FINISHED
hiatus -> MangaState.PAUSED
else -> MangaState.ONGOING
},
)
}
private fun parseChapters(chapterListGroup: JSONArray, language: String): List<MangaChapter> {
val chapterList = chapterListGroup
.toJSONList()
.flatMap {
it.optJSONArray("firstChapterList")?.toJSONList().orEmpty() +
it.optJSONArray("lastChapterList")?.toJSONList().orEmpty()
}
return chapterList.mapChapters { _, chapter ->
val chapterId = chapter.getInt("chapterId").toString()
val subtitle = chapter.getStringOrNull("subTitle") ?: return@mapChapters null
MangaChapter(
id = generateUid(chapterId),
url = chapterId,
name = subtitle,
number = chapter.getString("name")
.substringAfter("#")
.toIntOrNull() ?: -1,
uploadDate = chapter.getInt("startTimeStamp") * 1000L,
branch = when (language) {
"PORTUGUESE_BR" -> "Portuguese (Brazil)"
else -> language.lowercase().toTitleCase()
},
scanlator = null,
source = source,
)
}
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val pages = apiCall("/manga_viewer?chapter_id=${chapter.url}&split=yes&img_quality=super_high")
.getJSONObject("mangaViewer")
.getJSONArray("pages")
return pages.mapJSONNotNull {
val mangaPage = it.optJSONObject("mangaPage")
?: return@mapJSONNotNull null
val url = mangaPage.getString("imageUrl")
val encryptionKey = mangaPage.getStringOrNull("encryptionKey")
MangaPage(
id = generateUid(url),
url = url + if (encryptionKey == null) "" else "#$encryptionKey",
preview = null,
source = source,
)
}
}
// image descrambling
override fun intercept(chain: Interceptor.Chain): Response {
val request = chain.request()
val response = chain.proceed(request)
val encryptionKey = request.url.fragment
if (encryptionKey.isNullOrEmpty()) {
return response
}
val contentType = response.headers["Content-Type"] ?: "image/jpeg"
val image = requireNotNull(response.body).bytes().decodeXorCipher(encryptionKey)
val body = image.toResponseBody(contentType.toMediaTypeOrNull())
return response.newBuilder()
.body(body)
.build()
}
private fun ByteArray.decodeXorCipher(key: String): ByteArray {
val keyStream = key.chunked(2)
.map { it.toInt(16) }
return mapIndexed { i, byte -> byte.toInt() xor keyStream[i % keyStream.size] }
.map(Int::toByte)
.toByteArray()
}
private suspend fun apiCall(url: String): JSONObject {
val newUrl = "$apiUrl$url".toHttpUrl().newBuilder()
.addQueryParameter("format", "json")
.build()
val response = webClient.httpGet(newUrl, extraHeaders).parseJson()
val success = response.optJSONObject("success")
return checkNotNull(success) {
val error = response.getJSONObject("error")
val reason = error.getJSONArray("popups")
.toJSONList()
.firstOrNull { it.getStringOrNull("language") == null }
if (reason?.getStringOrNull("subject") == "Not Found" && url.contains("manga_viewer")) {
"This chapter has expired"
} else {
reason?.getStringOrNull("body") ?: "Unknown Error"
}
}
}
@MangaSourceParser("MANGAPLUSPARSER_EN", "MANGA Plus English", "en")
class English(context: MangaLoaderContext) : MangaPlusParser(
context,
MangaSource.MANGAPLUSPARSER_EN,
"ENGLISH",
)
@MangaSourceParser("MANGAPLUSPARSER_ES", "MANGA Plus Spanish", "es")
class Spanish(context: MangaLoaderContext) : MangaPlusParser(
context,
MangaSource.MANGAPLUSPARSER_ES,
"SPANISH",
)
@MangaSourceParser("MANGAPLUSPARSER_FR", "MANGA Plus French", "fr")
class French(context: MangaLoaderContext) : MangaPlusParser(
context,
MangaSource.MANGAPLUSPARSER_FR,
"FRENCH",
)
@MangaSourceParser("MANGAPLUSPARSER_ID", "MANGA Plus Indonesian", "id")
class Indonesian(context: MangaLoaderContext) : MangaPlusParser(
context,
MangaSource.MANGAPLUSPARSER_ID,
"INDONESIAN",
)
@MangaSourceParser("MANGAPLUSPARSER_PTBR", "MANGA Plus Portuguese (Brazil)", "pt")
class Portuguese(context: MangaLoaderContext) : MangaPlusParser(
context,
MangaSource.MANGAPLUSPARSER_PTBR,
"PORTUGUESE_BR",
)
@MangaSourceParser("MANGAPLUSPARSER_RU", "MANGA Plus Russian", "ru")
class Russian(context: MangaLoaderContext) : MangaPlusParser(
context,
MangaSource.MANGAPLUSPARSER_RU,
"RUSSIAN",
)
@MangaSourceParser("MANGAPLUSPARSER_TH", "MANGA Plus Thai", "th")
class Thai(context: MangaLoaderContext) : MangaPlusParser(
context,
MangaSource.MANGAPLUSPARSER_TH,
"THAI",
)
@MangaSourceParser("MANGAPLUSPARSER_VI", "MANGA Plus Vietnamese", "vi")
class Vietnamese(context: MangaLoaderContext) : MangaPlusParser(
context,
MangaSource.MANGAPLUSPARSER_VI,
"VIETNAMESE",
)
}

View File

@@ -1,7 +1,11 @@
package org.koitharu.kotatsu.parsers.site
package org.koitharu.kotatsu.parsers.site.all
import okhttp3.Headers
import androidx.collection.ArrayMap
import kotlinx.coroutines.sync.Mutex
import kotlinx.coroutines.sync.withLock
import okhttp3.HttpUrl.Companion.toHttpUrl
import okhttp3.Interceptor
import okhttp3.Response
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
@@ -12,64 +16,92 @@ import java.text.SimpleDateFormat
import java.util.*
internal abstract class NineMangaParser(
final override val context: MangaLoaderContext,
context: MangaLoaderContext,
source: MangaSource,
defaultDomain: String,
) : PagedMangaParser(source, pageSize = 26) {
) : PagedMangaParser(context, source, pageSize = 26), Interceptor {
override val configKeyDomain = ConfigKey.Domain(defaultDomain, null)
override val configKeyDomain = ConfigKey.Domain(defaultDomain)
init {
context.cookieJar.insertCookies(getDomain(), "ninemanga_template_desk=yes")
context.cookieJar.insertCookies(domain, "ninemanga_template_desk=yes")
}
override val headers = Headers.Builder()
override val headers = super.headers.newBuilder()
.add("Accept-Language", "en-US;q=0.7,en;q=0.3")
.build()
override val sortOrders: Set<SortOrder> = Collections.singleton(
override val availableSortOrders: Set<SortOrder> = Collections.singleton(
SortOrder.POPULARITY,
)
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
override val availableStates: Set<MangaState> = EnumSet.of(
MangaState.ONGOING,
MangaState.FINISHED,
)
override val isTagsExclusionSupported: Boolean = true
override fun intercept(chain: Interceptor.Chain): Response {
val request = chain.request()
val newRequest = if (request.url.host == domain) {
request.newBuilder().removeHeader("Referer").build()
} else {
request
}
return chain.proceed(newRequest)
}
override suspend fun getListPage(page: Int, filter: MangaListFilter?): List<Manga> {
val url = buildString {
append("https://")
append(getDomain())
when {
!query.isNullOrEmpty() -> {
append(domain)
when (filter) {
is MangaListFilter.Search -> {
append("/search/?name_sel=&wd=")
append(query.urlEncoded())
append(filter.query.urlEncoded())
append("&page=")
append(page)
append(".html")
}
!tags.isNullOrEmpty() -> {
append("/search/?category_id=")
for (tag in tags) {
append(tag.key)
append(',')
is MangaListFilter.Advanced -> {
if (filter.tags.isNotEmpty() || filter.tagsExclude.isNotEmpty() || filter.states.isNotEmpty()) {
append("/search/?category_id=")
append(filter.tags.joinToString(separator = ",") { it.key })
append("&out_category_id=")
append(filter.tagsExclude.joinToString(separator = ",") { it.key })
filter.states.oneOrThrowIfMany()?.let {
append("&completed_series=")
when (it) {
MangaState.ONGOING -> append("no")
MangaState.FINISHED -> append("yes")
else -> append("either")
}
}
append("&page=")
} else {
append("/category/index_")
}
append("&page=")
append(page.toString())
append(".html")
}
else -> {
null -> {
append("/category/index_")
append(page)
append(".html")
}
}
append(page)
append(".html")
}
val doc = context.httpGet(url, headers).parseHtml()
val root = doc.body().selectFirst("ul.direlist")
?: doc.parseFailed("Cannot find root")
val doc = webClient.httpGet(url).parseHtml()
val root = doc.body().selectFirst("ul.direlist") ?: doc.parseFailed("Cannot find root")
val baseHost = root.baseUri().toHttpUrl().host
return root.select("li").map { node ->
val href = node.selectFirst("a")?.absUrl("href")
?: node.parseFailed("Link not found")
val href = node.selectFirst("a")?.absUrl("href") ?: node.parseFailed("Link not found")
val relUrl = href.toRelativeUrl(baseHost)
val dd = node.selectFirst("dd")
Manga(
@@ -91,30 +123,25 @@ internal abstract class NineMangaParser(
}
override suspend fun getDetails(manga: Manga): Manga {
val doc = context.httpGet(
manga.url.toAbsoluteUrl(getDomain()) + "?waring=1",
headers,
val doc = webClient.httpGet(
manga.url.toAbsoluteUrl(domain) + "?waring=1",
).parseHtml()
val root = doc.body().selectFirstOrThrow("div.manga")
val infoRoot = root.selectFirstOrThrow("div.bookintro")
val tagMap = getOrCreateTagMap()
val selectTag = infoRoot.getElementsByAttributeValue("itemprop", "genre").first()?.select("a")
val tags = selectTag?.mapNotNullToSet { tagMap[it.text()] }
return manga.copy(
tags = infoRoot.getElementsByAttributeValue("itemprop", "genre").first()
?.select("a")?.mapToSet { a ->
MangaTag(
title = a.text().toTitleCase(),
key = a.attr("href").substringBetween("/", "."),
source = source,
)
}.orEmpty(),
tags = tags.orEmpty(),
author = infoRoot.getElementsByAttributeValue("itemprop", "author").first()?.text(),
state = parseStatus(infoRoot.select("li a.red").text()),
description = infoRoot.getElementsByAttributeValue("itemprop", "description").first()
?.html()?.substringAfter("</b>"),
description = infoRoot.getElementsByAttributeValue("itemprop", "description").first()?.html()
?.substringAfter("</b>"),
chapters = root.selectFirst("div.chapterbox")?.select("ul.sub_vol_ul > li")
?.asReversed()?.mapChapters { i, li ->
?.mapChapters(reversed = true) { i, li ->
val a = li.selectFirst("a.chapter_list_a")
val href = a?.attrAsRelativeUrlOrNull("href")
?.replace("%20", " ") ?: li.parseFailed("Link not found")
val href =
a?.attrAsRelativeUrlOrNull("href")?.replace("%20", " ") ?: li.parseFailed("Link not found")
MangaChapter(
id = generateUid(href),
name = a.text(),
@@ -130,13 +157,12 @@ internal abstract class NineMangaParser(
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val doc = context.httpGet(chapter.url.toAbsoluteUrl(getDomain()), headers).parseHtml()
val doc = webClient.httpGet(chapter.url.toAbsoluteUrl(domain)).parseHtml()
return doc.body().getElementById("page")?.select("option")?.map { option ->
val url = option.attr("value")
MangaPage(
id = generateUid(url),
url = url,
referer = chapter.url.toAbsoluteUrl(getDomain()),
preview = null,
source = source,
)
@@ -144,30 +170,58 @@ internal abstract class NineMangaParser(
}
override suspend fun getPageUrl(page: MangaPage): String {
val doc = context.httpGet(page.url.toAbsoluteUrl(getDomain()), headers).parseHtml()
val doc = webClient.httpGet(page.url.toAbsoluteUrl(domain)).parseHtml()
val root = doc.body()
return root.selectFirst("a.pic_download")?.absUrl("href")
?: doc.parseFailed("Page image not found")
return root.selectFirst("a.pic_download")?.absUrl("href") ?: doc.parseFailed("Page image not found")
}
override suspend fun getTags(): Set<MangaTag> {
val doc = context.httpGet("https://${getDomain()}/search/?type=high", headers)
.parseHtml()
val root = doc.body().getElementById("search_form")
return root?.select("li.cate_list")?.mapNotNullToSet { li ->
val cateId = li.attr("cate_id") ?: return@mapNotNullToSet null
val a = li.selectFirst("a") ?: return@mapNotNullToSet null
MangaTag(
title = a.text().toTitleCase(),
private var tagCache: ArrayMap<String, MangaTag>? = null
private val mutex = Mutex()
override suspend fun getAvailableTags(): Set<MangaTag> {
return getOrCreateTagMap().values.toSet()
}
private suspend fun getOrCreateTagMap(): Map<String, MangaTag> = mutex.withLock {
tagCache?.let { return@withLock it }
val tagMap = ArrayMap<String, MangaTag>()
val tagElements = webClient.httpGet("https://${domain}/search/?type=high").parseHtml().select("li.cate_list")
for (el in tagElements) {
if (el.text().isEmpty()) continue
val cateId = el.attr("cate_id")
val a = el.selectFirstOrThrow("a")
tagMap[el.text()] = MangaTag(
title = a.text().toTitleCase(sourceLocale),
key = cateId,
source = source,
)
} ?: doc.parseFailed("Root not found")
}
tagCache = tagMap
return@withLock tagMap
}
private fun parseStatus(status: String) = when {
// en
status.contains("Ongoing") -> MangaState.ONGOING
status.contains("Completed") -> MangaState.FINISHED
//es
status.contains("En curso") -> MangaState.ONGOING
status.contains("Completado") -> MangaState.FINISHED
//ru
status.contains("постоянный") -> MangaState.ONGOING
status.contains("завершенный") -> MangaState.FINISHED
//de
status.contains("Laufende") -> MangaState.ONGOING
status.contains("Abgeschlossen") -> MangaState.FINISHED
//pt
status.contains("Completo") -> MangaState.ONGOING
status.contains("Em tradução") -> MangaState.FINISHED
//it
status.contains("In corso") -> MangaState.ONGOING
status.contains("Completato") -> MangaState.FINISHED
//fr
status.contains("En cours") -> MangaState.ONGOING
status.contains("Complété") -> MangaState.FINISHED
else -> null
}
@@ -257,4 +311,4 @@ internal abstract class NineMangaParser(
MangaSource.NINEMANGA_FR,
"fr.ninemanga.com",
)
}
}

View File

@@ -0,0 +1,387 @@
package org.koitharu.kotatsu.parsers.site.all
import okhttp3.HttpUrl.Companion.toHttpUrlOrNull
import okhttp3.Interceptor
import okhttp3.Response
import org.json.JSONArray
import org.json.JSONObject
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import org.koitharu.kotatsu.parsers.util.json.getStringOrNull
import org.koitharu.kotatsu.parsers.util.json.mapJSON
import org.koitharu.kotatsu.parsers.util.json.mapJSONToSet
import org.koitharu.kotatsu.parsers.util.json.toJSONList
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("NINENINENINEHENTAI", "999Hentai", type = ContentType.HENTAI)
internal class NineNineNineHentaiParser(context: MangaLoaderContext) :
PagedMangaParser(context, MangaSource.NINENINENINEHENTAI, size), Interceptor {
override val configKeyDomain = ConfigKey.Domain("999hentai.net")
override val availableSortOrders: EnumSet<SortOrder> = EnumSet.of(
SortOrder.POPULARITY,
SortOrder.NEWEST,
)
override val isMultipleTagsSupported = false
override suspend fun getAvailableLocales() = setOf(
Locale.ENGLISH,
Locale.CHINESE,
Locale.JAPANESE,
Locale("es"),
)
private fun Locale?.getSiteLang(): String {
if (this == null) return "all"
return when {
equals(Locale.ENGLISH) -> "en"
equals(Locale.CHINESE) -> "cn"
equals(Locale.JAPANESE) -> "jp"
equals(Locale("es")) -> "es"
else -> "all"
}
}
// Need for disable encoding (with encoding not working)
override fun intercept(chain: Interceptor.Chain): Response {
val request = chain.request()
val newRequest = if (request.header("Content-Encoding") != null) {
request.newBuilder().removeHeader("Content-Encoding").build()
} else {
request
}
return chain.proceed(newRequest)
}
private val cdnHost = SuspendLazy(::getUpdatedCdnHost)
private suspend fun getUpdatedCdnHost(): String {
val url = "https://$domain/manga-home"
val response = webClient.httpGet(url).parseHtml()
val cdn = response.selectFirst("img.v-thumbnail")?.attr("data-src")
return cdn?.toHttpUrlOrNull()?.host ?: "edge.fast4speed.rsvp"
}
override suspend fun getAvailableTags(): Set<MangaTag> {
val query = """
queryTags(
search: {format:"tagchapter",sortBy:Popular}
page: 1
limit: 100
) {
edges {
name
}
}
""".trimIndent()
val tags = apiCall(query)
.getJSONObject("queryTags")
.getJSONArray("edges")
return tags.mapJSONToSet {
val name = it.getString("name")
MangaTag(
title = name.toCamelCase(),
key = name,
source = source,
)
}
}
override suspend fun getListPage(page: Int, filter: MangaListFilter?): List<Manga> {
return when (filter) {
is MangaListFilter.Advanced -> {
if (filter.tags.isEmpty() && filter.sortOrder == SortOrder.POPULARITY) {
getPopularList(page, filter.locale)
} else {
getSearchList(page, null, filter.locale, filter.tags, filter.sortOrder)
}
}
is MangaListFilter.Search -> {
getSearchList(page, filter.query, null, null, filter.sortOrder)
}
else -> {
getPopularList(page, null)
}
}
}
private suspend fun getPopularList(
page: Int,
locale: Locale?,
): List<Manga> {
val query = """
queryPopularChapters(
size: $size
language: "${locale.getSiteLang()}"
dateRange: 1
page: $page
) {
edges {
_id
name
firstPics
}
}
""".trimIndent()
return apiCall(query)
.getJSONObject("queryPopularChapters")
.getJSONArray("edges")
.toMangaList()
}
private suspend fun getSearchList(
page: Int,
search: String?,
locale: Locale?,
tags: Set<MangaTag>?,
sort: SortOrder?,
): List<Manga> {
val searchPayload = buildString {
if (!search.isNullOrEmpty()) {
append("query:\"$search\",")
}
append("language:\"${locale.getSiteLang()}\"")
if (sort == SortOrder.POPULARITY) {
append(",sortBy:Popular")
}
if (!tags.isNullOrEmpty()) {
val tag = tags.oneOrThrowIfMany()!!.key
append(",tags:[\"$tag\"]")
}
}
val query = """
queryChapters(
limit: $size
search: {$searchPayload}
page: $page
) {
edges {
_id
name
firstPics
}
}
""".trimIndent()
return apiCall(query)
.getJSONObject("queryChapters")
.getJSONArray("edges")
.toMangaList()
}
private suspend fun JSONArray.toMangaList(): List<Manga> = mapJSON { entry ->
val id = entry.getString("_id")
val name = entry.getString("name")
val cover = runCatching {
entry.getJSONArray("firstPics")
.getJSONObject(0)
.getString("url")
}.getOrNull()
Manga(
id = generateUid(id),
title = name.replace(shortenTitleRegex, "").trim(),
altTitle = name,
coverUrl = when {
cover?.startsWith("http") == true -> cover
cover == null -> ""
else -> "https://${cdnHost.get()}/$cover"
},
author = null,
isNsfw = true,
url = id,
publicUrl = "/hchapter/$id".toAbsoluteUrl(domain),
tags = emptySet(),
source = source,
state = null,
rating = RATING_UNKNOWN,
)
}
override suspend fun getDetails(manga: Manga): Manga {
val query = """
queryChapter(
chapterId: "${manga.url}"
) {
_id
name
uploadDate
format
description
language
pages
tags
pictureUrls {
picCdn
pics
picsS
}
}
""".trimIndent()
val entry = apiCall(query)
.getJSONObject("queryChapter")
val id = entry.getString("_id")
val name = entry.getString("name")
val cover = entry.getJSONArray("pictureUrls")
.getJSONObject(0)
.let { pics ->
val cdn = pics.getString("picCdn").let {
if (it.startsWith("http")) {
"$it/"
} else {
"https://${cdnHost.get()}/$it/"
}
}
val img = pics.getJSONArray("pics").getJSONObject(0).getString("url")
val imgS = pics.getJSONArray("picsS").getJSONObject(0).getString("url")
Pair(cdn + imgS, cdn + img)
}
val tags = entry.optJSONArray("tags")?.mapJSON {
SiteTag(
name = it.getString("tagName"),
type = it.getStringOrNull("tagType"),
)
}
return manga.copy(
title = name.replace(shortenTitleRegex, "").trim(),
altTitle = name,
coverUrl = cover.first,
largeCoverUrl = cover.second,
author = tags?.filter { it.type == "artist" }?.joinToString { it.name.toCamelCase() },
isNsfw = true,
tags = tags?.mapToSet {
MangaTag(
title = it.name.toCamelCase(),
key = it.name,
source = source,
)
}.orEmpty(),
state = null,
description = entry.getStringOrNull("description"),
chapters = listOf(
MangaChapter(
id = generateUid(id),
name = name,
number = 1,
url = id,
uploadDate = runCatching {
dateFormat.parse(entry.getString("uploadDate"))!!.time
}.getOrDefault(0L),
branch = entry.getStringOrNull("language")?.let {
val locale = when (it) {
"en" -> Locale.ENGLISH
"jp" -> Locale.JAPANESE
"cn" -> Locale.CHINESE
"es" -> Locale("es")
else -> Locale.ROOT
}
return@let locale.getDisplayLanguage(locale)
},
scanlator = when (entry.getStringOrNull("format")) {
"artistcg" -> "ArtistCG"
"gamecg" -> "GameCG"
"imageset" -> "ImageSet"
else -> entry.getStringOrNull("format")?.toCamelCase()
},
source = source,
),
),
)
}
private data class SiteTag(
val name: String,
val type: String?,
)
override suspend fun getRelatedManga(seed: Manga): List<Manga> {
val query = """
queryRecommendations(
type: "chapter"
_id: "${seed.url}"
search: {sortBy:Popular}
page: 1
size: $size
) {
chapters {
_id
name
firstPics
}
}
""".trimIndent()
return apiCall(query)
.getJSONObject("queryRecommendations")
.getJSONArray("chapters")
.toMangaList()
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val query = """
queryChapter(
chapterId: "${chapter.url}"
) {
pictureUrls {
picCdn
pics
picsS
}
}
""".trimIndent()
val pages = apiCall(query)
.getJSONObject("queryChapter")
.getJSONArray("pictureUrls")
.getJSONObject(0)
val cdn = pages.getString("picCdn").let {
if (it.startsWith("http")) {
"$it/"
} else {
"https://${cdnHost.get()}/$it/"
}
}
val pics = pages.getJSONArray("pics").toJSONList()
val picsS = pages.getJSONArray("picsS").toJSONList()
return pics.zip(picsS).map {
val img = it.first.getString("url")
val imgS = it.second.getString("url")
MangaPage(
id = generateUid(img),
url = cdn + img,
preview = cdn + imgS,
source = source,
)
}
}
private suspend fun apiCall(query: String): JSONObject {
return webClient.graphQLQuery("https://api.$domain/api", query).getJSONObject("data")
}
companion object {
private const val size = 20
private val shortenTitleRegex = Regex("""(\[[^]]*]|[({][^)}]*[)}])""")
private val dateFormat = SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'", Locale.ENGLISH)
}
}

View File

@@ -0,0 +1,358 @@
package org.koitharu.kotatsu.parsers.site.all
import kotlinx.coroutines.async
import kotlinx.coroutines.awaitAll
import kotlinx.coroutines.coroutineScope
import okhttp3.Headers
import okhttp3.HttpUrl
import okhttp3.HttpUrl.Companion.toHttpUrl
import org.json.JSONArray
import org.json.JSONObject
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaParser
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.exception.NotFoundException
import org.koitharu.kotatsu.parsers.exception.ParseException
import org.koitharu.kotatsu.parsers.model.ContentType
import org.koitharu.kotatsu.parsers.model.Manga
import org.koitharu.kotatsu.parsers.model.MangaChapter
import org.koitharu.kotatsu.parsers.model.MangaListFilter
import org.koitharu.kotatsu.parsers.model.MangaPage
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.model.MangaTag
import org.koitharu.kotatsu.parsers.model.RATING_UNKNOWN
import org.koitharu.kotatsu.parsers.model.SortOrder
import org.koitharu.kotatsu.parsers.util.SoftSuspendLazy
import org.koitharu.kotatsu.parsers.util.SuspendLazy
import org.koitharu.kotatsu.parsers.util.domain
import org.koitharu.kotatsu.parsers.util.generateUid
import org.koitharu.kotatsu.parsers.util.json.getBooleanOrDefault
import org.koitharu.kotatsu.parsers.util.json.getFloatOrDefault
import org.koitharu.kotatsu.parsers.util.json.getIntOrDefault
import org.koitharu.kotatsu.parsers.util.json.getStringOrNull
import org.koitharu.kotatsu.parsers.util.json.mapJSON
import org.koitharu.kotatsu.parsers.util.json.mapJSONIndexed
import org.koitharu.kotatsu.parsers.util.mapChapters
import org.koitharu.kotatsu.parsers.util.oneOrThrowIfMany
import org.koitharu.kotatsu.parsers.util.parseJson
import org.koitharu.kotatsu.parsers.util.splitTwoParts
import org.koitharu.kotatsu.parsers.util.toAbsoluteUrl
import org.koitharu.kotatsu.parsers.util.urlEncoded
import java.util.Calendar
import java.util.EnumSet
import javax.crypto.Mac
import javax.crypto.spec.SecretKeySpec
internal abstract class WebtoonsParser(
context: MangaLoaderContext,
source: MangaSource,
) : MangaParser(context, source) {
override val isMultipleTagsSupported = false
private val signer by lazy {
WebtoonsUrlSigner("gUtPzJFZch4ZyAGviiyH94P99lQ3pFdRTwpJWDlSGFfwgpr6ses5ALOxWHOIT7R1")
}
// we don't __really__ support changing this domain because:
// 1. I don't think other websites have this exact API
// 2. most communication is done with other domains (hosting API and static content), which are not configurable
// 3. we rely on the HTTP client setting the referer header to webtoons.com
//
// This effectively means that changing the domain will break the source. Yikes
override val configKeyDomain = ConfigKey.Domain("webtoons.com")
private val apiDomain = "global.apis.naver.com"
private val staticDomain = "webtoon-phinf.pstatic.net"
override val availableSortOrders: Set<SortOrder> = EnumSet.of(
SortOrder.POPULARITY, // views
SortOrder.RATING, // star rating
//SortOrder.LIKE, // likes
SortOrder.UPDATED,
)
override val headers: Headers
get() = Headers.Builder().add("User-Agent", "nApps (Android 12;; linewebtoon; 3.1.0)").build()
override suspend fun getPageUrl(page: MangaPage): String {
return page.url.toAbsoluteUrl(staticDomain)
}
// some language tags do not map perfectly to the ones used by the API
private val languageCode: String
get() = when (val tag = sourceLocale.toLanguageTag()) {
"in" -> "id"
"zh" -> "zh-hant"
else -> tag
}
private suspend fun fetchEpisodes(titleNo: Long): List<MangaChapter> = coroutineScope {
val firstResult =
makeRequest("/lineWebtoon/webtoon/episodeList.json?v=5&titleNo=$titleNo&startIndex=0&pageSize=30")
val totalEpisodeCount = firstResult.getJSONObject("episodeList").getInt("totalServiceEpisodeCount")
val episodes = firstResult.getJSONObject("episodeList").getJSONArray("episode").toJSONList().toMutableList()
val additionalEpisodes = (episodes.size until totalEpisodeCount step 30).map { startIndex ->
async {
makeRequest("/lineWebtoon/webtoon/episodeList.json?v=5&titleNo=$titleNo&startIndex=$startIndex&pageSize=30").getJSONObject(
"episodeList",
).getJSONArray("episode").toJSONList()
}
}.awaitAll().flatten()
episodes.addAll(additionalEpisodes)
// Optimize object creation and sorting
episodes.mapChapters { i, jo ->
MangaChapter(
id = generateUid("$titleNo-$i"),
name = jo.getString("episodeTitle"),
number = jo.getInt("episodeSeq"),
url = "$titleNo-${jo.get("episodeNo")}",
uploadDate = jo.getLong("registerYmdt"),
branch = null,
scanlator = null,
source = source,
)
}.sortedBy(MangaChapter::number)
}
private fun JSONArray.toJSONList(): List<JSONObject> {
val list = mutableListOf<JSONObject>()
for (i in 0 until length()) {
list.add(getJSONObject(i))
}
return list
}
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val titleNo = manga.url.toLong()
val chaptersDeferred = async { fetchEpisodes(titleNo) }
val chapters = chaptersDeferred.await()
makeRequest("/lineWebtoon/webtoon/titleInfo.json?titleNo=${titleNo}&anyServiceStatus=false").getJSONObject("titleInfo")
.let { jo ->
MangaWebtoon(
Manga(
id = generateUid(titleNo),
title = jo.getString("title"),
altTitle = null,
url = "$titleNo",
publicUrl = "https://$domain/$languageCode/originals/a/list?title_no=${titleNo}",
rating = jo.getFloatOrDefault("starScoreAverage", -10f) / 10f,
isNsfw = jo.getBooleanOrDefault("ageGradeNotice", isNsfwSource),
coverUrl = jo.getString("thumbnail").toAbsoluteUrl(staticDomain),
largeCoverUrl = jo.getStringOrNull("thumbnailVertical")?.toAbsoluteUrl(staticDomain),
tags = setOf(parseTag(jo.getJSONObject("genreInfo"))),
author = jo.getStringOrNull("writingAuthorName"),
description = jo.getString("synopsis"),
// I don't think the API provides this info,
state = null,
chapters = chapters,
source = source,
),
date = jo.getLong("lastEpisodeRegisterYmdt"),
readCount = jo.getLong("readCount"),
//likeCount = jo.getLong("likeitCount"),
).manga
}
}
private val allGenreCache = SuspendLazy {
makeRequest("/lineWebtoon/webtoon/genreList.json").getJSONObject("genreList").getJSONArray("genres")
.mapJSON { jo -> parseTag(jo) }.associateBy { tag -> tag.key }
}
private val allTitleCache = SoftSuspendLazy {
makeRequest("/lineWebtoon/webtoon/titleList.json?").getJSONObject("titleList").getJSONArray("titles")
.mapJSON { jo ->
val titleNo = jo.getLong("titleNo")
MangaWebtoon(
Manga(
id = generateUid(titleNo),
url = titleNo.toString(),
publicUrl = "https://$domain/$languageCode/originals/a/list?title_no=$titleNo",
title = jo.getString("title"),
coverUrl = jo.getString("thumbnail").toAbsoluteUrl(staticDomain),
altTitle = null,
author = jo.getStringOrNull("writingAuthorName"),
isNsfw = jo.getBooleanOrDefault("ageGradeNotice", isNsfwSource),
rating = jo.getFloatOrDefault("starScoreAverage", -10f) / 10f,
tags = setOfNotNull(allGenreCache.get()[jo.getString("representGenre")]),
description = jo.getString("synopsis"),
state = null,
source = source,
),
date = jo.getLong("lastEpisodeRegisterYmdt"),
readCount = jo.getLong("readCount"),
//likeCount = jo.getLong("likeitCount"),
)
}
}
private suspend fun getAllGenreList(): Map<String, MangaTag> {
return allGenreCache.get()
}
private suspend fun getAllTitleList(): List<MangaWebtoon> {
return allTitleCache.get()
}
override suspend fun getList(offset: Int, filter: MangaListFilter?): List<Manga> {
val webtoons = when (filter) {
is MangaListFilter.Search -> {
makeRequest("/lineWebtoon/webtoon/searchWebtoon?query=${filter.query.urlEncoded()}").getJSONObject("webtoonSearch")
.getJSONArray("titleList").mapJSON { jo ->
val titleNo = jo.getLong("titleNo")
MangaWebtoon(
Manga(
id = generateUid(titleNo),
title = jo.getString("title"),
altTitle = null,
url = titleNo.toString(),
publicUrl = "https://$domain/$languageCode/originals/a/list?title_no=$titleNo",
rating = RATING_UNKNOWN,
isNsfw = isNsfwSource,
coverUrl = jo.getString("thumbnail").toAbsoluteUrl(staticDomain),
largeCoverUrl = null,
tags = emptySet(),
author = jo.getStringOrNull("writingAuthorName"),
description = null,
state = null,
source = source,
))
}
}
is MangaListFilter.Advanced -> {
val genre = filter.tags.oneOrThrowIfMany()?.key ?: "ALL"
val genres = getAllGenreList()
val result = getAllTitleList()
val sortedResult = when (filter.sortOrder) {
SortOrder.UPDATED -> result.sortedBy { it.date }
SortOrder.POPULARITY -> result.sortedByDescending { it.readCount }
SortOrder.RATING -> result.sortedByDescending { it.manga.rating }
//SortOrder.LIKE -> result.sortedBy { it.likeitCount }
else -> throw IllegalArgumentException("Unsupported sort order: ${filter.sortOrder}")
}
if (genre != "ALL") {
sortedResult.filter { it.manga.tags.contains(genres[genre]) }
} else {
sortedResult
}
}
else -> getAllTitleList()
}
return webtoons.map { it.manga }.subList(offset, (offset + 20).coerceAtMost(webtoons.size))
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val (titleNo, episodeNo) = requireNotNull(chapter.url.splitTwoParts('-'))
return makeRequest("/lineWebtoon/webtoon/episodeInfo.json?v=4&titleNo=$titleNo&episodeNo=$episodeNo").getJSONObject(
"episodeInfo",
).getJSONArray("imageInfo").mapJSONIndexed { i, jo ->
MangaPage(
id = generateUid("$titleNo-$episodeNo-$i"),
url = jo.getString("url"),
preview = null,
source = source,
)
}
}
private fun parseTag(jo: JSONObject): MangaTag {
return MangaTag(
title = jo.getString("name"),
key = jo.getString("code"),
source = source,
)
}
override suspend fun getAvailableTags(): Set<MangaTag> {
return getAllGenreList().values.toSet()
}
private suspend fun makeRequest(url: String): JSONObject {
val resp = webClient.httpGet(finalizeUrl(url))
val message: JSONObject? = resp.parseJson().optJSONObject("message")
return when (resp.code) {
in 200..299 -> checkNotNull(message).getJSONObject("result")
404 -> throw NotFoundException(message?.getStringOrNull("message").orEmpty(), url)
else -> {
val code = message?.getIntOrDefault("code", 0)
val errorMessage = message?.getStringOrNull("message")
throw ParseException("Api error (code=$code): $errorMessage", url)
}
}
}
private fun finalizeUrl(url: String): HttpUrl {
val httpUrl = url.toAbsoluteUrl(apiDomain).toHttpUrl()
val builder = httpUrl.newBuilder().addQueryParameter("serviceZone", "GLOBAL")
if (httpUrl.queryParameter("v") == null) {
builder.addQueryParameter("v", "1")
}
builder.addQueryParameter("language", languageCode).addQueryParameter("locale", "languageCode")
.addQueryParameter("platform", "APP_ANDROID")
signer.makeEncryptUrl(builder)
return builder.build()
}
@MangaSourceParser("WEBTOONS_EN", "Webtoons English", "en", type = ContentType.MANGA)
class English(context: MangaLoaderContext) : WebtoonsParser(context, MangaSource.WEBTOONS_EN)
@MangaSourceParser("WEBTOONS_ID", "Webtoons Indonesia", "id", type = ContentType.MANGA)
class Indonesian(context: MangaLoaderContext) : WebtoonsParser(context, MangaSource.WEBTOONS_ID)
@MangaSourceParser("WEBTOONS_ES", "Webtoons Spanish", "es", type = ContentType.MANGA)
class Spanish(context: MangaLoaderContext) : WebtoonsParser(context, MangaSource.WEBTOONS_ES)
@MangaSourceParser("WEBTOONS_FR", "Webtoons French", "fr", type = ContentType.MANGA)
class French(context: MangaLoaderContext) : WebtoonsParser(context, MangaSource.WEBTOONS_FR)
@MangaSourceParser("WEBTOONS_TH", "Webtoons Thai", "th", type = ContentType.MANGA)
class Thai(context: MangaLoaderContext) : WebtoonsParser(context, MangaSource.WEBTOONS_TH)
@MangaSourceParser("WEBTOONS_ZH", "Webtoons Chinese", "zh", type = ContentType.MANGA)
class Chinese(context: MangaLoaderContext) : LineWebtoonsParser(context, MangaSource.WEBTOONS_ZH)
@MangaSourceParser("WEBTOONS_DE", "Webtoons German", "de", type = ContentType.MANGA)
class German(context: MangaLoaderContext) : LineWebtoonsParser(context, MangaSource.WEBTOONS_DE)
private inner class WebtoonsUrlSigner(private val secret: String) {
private val mac = Mac.getInstance("HmacSHA1").apply {
this.init(SecretKeySpec(secret.encodeToByteArray(), "HmacSHA1"))
}
private fun getMessage(url: String, msgpad: String): String {
return url.substring(0, 0xFF.coerceAtMost(url.length)) + msgpad
}
private fun getMessageDigest(s: String): String {
val signedMessage = synchronized(mac) { mac.doFinal(s.toByteArray()) }
return context.encodeBase64(signedMessage)
}
fun makeEncryptUrl(urlBuilder: HttpUrl.Builder) {
val msgPad = Calendar.getInstance().timeInMillis.toString()
val digest = getMessageDigest(getMessage(urlBuilder.build().toString(), msgPad))
urlBuilder.addQueryParameter("msgpad", msgPad).addQueryParameter("md", digest)
// .addEncodedQueryParameter("md", digest.urlEncoded())
}
}
private inner class MangaWebtoon(
val manga: Manga,
@JvmField val date: Long? = null,
@JvmField val readCount: Long? = null,
)
}

View File

@@ -0,0 +1,198 @@
package org.koitharu.kotatsu.parsers.site.animebootstrap
import kotlinx.coroutines.async
import kotlinx.coroutines.coroutineScope
import org.json.JSONArray
import org.jsoup.nodes.Document
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.util.*
internal abstract class AnimeBootstrapParser(
context: MangaLoaderContext,
source: MangaSource,
domain: String,
pageSize: Int = 24,
) : PagedMangaParser(context, source, pageSize) {
override val configKeyDomain = ConfigKey.Domain(domain)
override val isMultipleTagsSupported = false
override val availableSortOrders: Set<SortOrder> = EnumSet.of(
SortOrder.UPDATED,
SortOrder.POPULARITY,
SortOrder.ALPHABETICAL,
SortOrder.NEWEST,
)
protected open val listUrl = "/manga"
protected open val datePattern = "dd MMM. yyyy"
init {
paginator.firstPage = 1
searchPaginator.firstPage = 1
}
override suspend fun getListPage(page: Int, filter: MangaListFilter?): List<Manga> {
val url = buildString {
append("https://")
append(domain)
append(listUrl)
append("?page=")
append(page.toString())
append("&type=all")
when (filter) {
is MangaListFilter.Search -> {
append("&search=")
append(filter.query.urlEncoded())
}
is MangaListFilter.Advanced -> {
filter.tags.oneOrThrowIfMany()?.let {
append("&categorie=")
append(it.key)
}
append("&sort=")
when (filter.sortOrder) {
SortOrder.POPULARITY -> append("view")
SortOrder.UPDATED -> append("updated")
SortOrder.ALPHABETICAL -> append("default")
SortOrder.NEWEST -> append("published")
else -> append("updated")
}
}
null -> append("&sort=updated")
}
}
val doc = webClient.httpGet(url).parseHtml()
return doc.select("div.col-6 div.product__item").map { div ->
val href = div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(div.host ?: domain),
coverUrl = div.selectFirstOrThrow("div.product__item__pic").attr("data-setbg").orEmpty(),
title = div.selectFirstOrThrow("div.product__item__text").text().orEmpty(),
altTitle = null,
rating = RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = null,
source = source,
isNsfw = isNsfwSource,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
val doc = webClient.httpGet("https://$domain$listUrl").parseHtml()
return doc.select("div.product__page__filter div:contains(Genre:) option ").mapNotNullToSet { option ->
val key = option.attr("value") ?: return@mapNotNullToSet null
val name = option.text().toTitleCase(sourceLocale)
MangaTag(
key = key,
title = name,
source = source,
)
}
}
protected open val selectDesc = "div.anime__details__text p"
protected open val selectState = "div.anime__details__widget li:contains(Ongoing)"
protected open val selectTag = "div.anime__details__widget li:contains(Categorie) a"
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val fullUrl = manga.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
val chaptersDeferred = async { getChapters(doc) }
val desc = doc.selectFirstOrThrow(selectDesc).html()
val state = if (doc.select(selectState).isNullOrEmpty()) {
MangaState.FINISHED
} else {
MangaState.ONGOING
}
manga.copy(
tags = doc.body().select(selectTag).mapNotNullToSet { a ->
MangaTag(
key = a.attr("href").substringAfterLast('='),
title = a.text().toTitleCase().replace(",", ""),
source = source,
)
},
description = desc,
state = state,
chapters = chaptersDeferred.await(),
)
}
protected open val selectChapter = "div.anime__details__episodes a"
protected open suspend fun getChapters(doc: Document): List<MangaChapter> {
return doc.body().select(selectChapter).mapChapters(reversed = true) { i, a ->
val href = a.attr("href")
MangaChapter(
id = generateUid(href),
name = a.text(),
number = i + 1,
url = href,
uploadDate = 0,
source = source,
scanlator = null,
branch = null,
)
}
}
protected open val selectPage = "div.read-img img"
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
if (doc.select("script:containsData(page_image)").isNullOrEmpty()) {
return doc.select(selectPage).map { img ->
val url = img.attr("onerror").replace("this.onerror=null;this.src=`", "").replace("`;", "")
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
)
}
} else {
val script = doc.selectFirstOrThrow("script:containsData(page_image)")
val images = JSONArray(script.data().substringAfterLast("var pages = ").substringBefore(';'))
val pages = ArrayList<MangaPage>(images.length())
for (i in 0 until images.length()) {
val pageTake = images.getJSONObject(i)
pages.add(
MangaPage(
id = generateUid(pageTake.getString("page_image")),
url = pageTake.getString("page_image"),
preview = null,
source = source,
),
)
}
return pages
}
}
}

View File

@@ -0,0 +1,138 @@
package org.koitharu.kotatsu.parsers.site.animebootstrap.fr
import kotlinx.coroutines.async
import kotlinx.coroutines.coroutineScope
import org.jsoup.nodes.Document
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.site.animebootstrap.AnimeBootstrapParser
import org.koitharu.kotatsu.parsers.util.*
import java.text.SimpleDateFormat
import java.util.EnumSet
import java.util.Locale
@MangaSourceParser("PAPSCAN", "PapScan", "fr")
internal class PapScan(context: MangaLoaderContext) :
AnimeBootstrapParser(context, MangaSource.PAPSCAN, "papscan.com") {
override val sourceLocale: Locale = Locale.ENGLISH
override val isMultipleTagsSupported = false
override val listUrl = "/liste-manga"
override val selectState = "div.anime__details__widget li:contains(En cours)"
override val selectTag = "div.anime__details__widget li:contains(Genre) a"
override val selectChapter = "ul.chapters li"
override val availableSortOrders: Set<SortOrder> = EnumSet.of(
SortOrder.POPULARITY,
SortOrder.ALPHABETICAL,
SortOrder.ALPHABETICAL_DESC,
)
override suspend fun getListPage(page: Int, filter: MangaListFilter?): List<Manga> {
val url = buildString {
append("https://")
append(domain)
append("/filterList")
append("?page=")
append(page.toString())
when (filter) {
is MangaListFilter.Search -> {
append("&alpha=")
append(filter.query.urlEncoded())
}
is MangaListFilter.Advanced -> {
filter.tags.oneOrThrowIfMany()?.let {
append("&cat=")
append(it.key)
}
append("&sortBy=")
when (filter.sortOrder) {
SortOrder.POPULARITY -> append("views")
SortOrder.ALPHABETICAL_DESC -> append("name&asc=false")
SortOrder.ALPHABETICAL -> append("name&asc=true")
else -> append("updated")
}
}
null -> append("&sortBy=updated")
}
}
val doc = webClient.httpGet(url).parseHtml()
return doc.select("div.product__item").map { div ->
val href = div.selectFirstOrThrow("h5 a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(div.host ?: domain),
coverUrl = div.selectFirstOrThrow("div.product__item__pic").attr("data-setbg").orEmpty(),
title = div.selectFirstOrThrow("div.product__item__text h5").text().orEmpty(),
altTitle = null,
rating = RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = null,
source = source,
isNsfw = isNsfwSource,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
val doc = webClient.httpGet("https://$domain$listUrl").parseHtml()
return doc.select("a.category ").mapNotNullToSet { a ->
val key = a.attr("href").substringAfterLast('=')
val name = a.text()
MangaTag(
key = key,
title = name,
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val fullUrl = manga.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
val chaptersDeferred = async { getChapters(doc) }
val desc = doc.selectFirstOrThrow(selectDesc).html()
val state = if (doc.select(selectState).isNullOrEmpty()) {
MangaState.FINISHED
} else {
MangaState.ONGOING
}
manga.copy(
tags = doc.body().select(selectTag).mapNotNullToSet { a ->
MangaTag(
key = a.attr("href").removeSuffix('/').substringAfterLast('/'),
title = a.text().toTitleCase(),
source = source,
)
},
description = desc,
state = state,
chapters = chaptersDeferred.await(),
)
}
override suspend fun getChapters(doc: Document): List<MangaChapter> {
val dateFormat = SimpleDateFormat(datePattern, sourceLocale)
return doc.body().select(selectChapter).mapChapters(reversed = true) { i, li ->
val href = li.selectFirstOrThrow("a").attr("href")
val dateText = li.selectFirst("span.date-chapter-title-rtl")?.text()
MangaChapter(
id = generateUid(href),
name = li.selectFirstOrThrow("span em").text(),
number = i + 1,
url = href,
uploadDate = dateFormat.tryParse(dateText),
source = source,
scanlator = null,
branch = null,
)
}
}
}

View File

@@ -0,0 +1,10 @@
package org.koitharu.kotatsu.parsers.site.animebootstrap.id
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.site.animebootstrap.AnimeBootstrapParser
@MangaSourceParser("KOMIKZOID", "KomikzoId", "id")
internal class KomikzoId(context: MangaLoaderContext) :
AnimeBootstrapParser(context, MangaSource.KOMIKZOID, "komikzoid.id")

View File

@@ -0,0 +1,10 @@
package org.koitharu.kotatsu.parsers.site.animebootstrap.id
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.site.animebootstrap.AnimeBootstrapParser
@MangaSourceParser("NEUMANGA", "NeuManga.xyz", "id")
internal class NeuManga(context: MangaLoaderContext) :
AnimeBootstrapParser(context, MangaSource.NEUMANGA, "neumanga.xyz")

View File

@@ -0,0 +1,10 @@
package org.koitharu.kotatsu.parsers.site.animebootstrap.id
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.site.animebootstrap.AnimeBootstrapParser
@MangaSourceParser("SEKTEKOMIK", "SekteKomik", "id")
internal class SekteKomik(context: MangaLoaderContext) :
AnimeBootstrapParser(context, MangaSource.SEKTEKOMIK, "sektekomik.xyz")

View File

@@ -0,0 +1,214 @@
package org.koitharu.kotatsu.parsers.site.ar
import kotlinx.coroutines.async
import kotlinx.coroutines.coroutineScope
import org.json.JSONArray
import org.json.JSONObject
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import org.koitharu.kotatsu.parsers.util.json.mapJSON
import org.koitharu.kotatsu.parsers.util.json.mapJSONIndexed
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("FLIXSCANS", "FlixScans.com", "ar")
internal class FlixScans(context: MangaLoaderContext) : PagedMangaParser(context, MangaSource.FLIXSCANS, 18) {
override val availableSortOrders: Set<SortOrder> = EnumSet.of(SortOrder.UPDATED)
override val availableStates: Set<MangaState> = EnumSet.allOf(MangaState::class.java)
override val availableContentRating: Set<ContentRating> = EnumSet.of(ContentRating.ADULT)
override val configKeyDomain = ConfigKey.Domain("flixscans.com")
override suspend fun getListPage(page: Int, filter: MangaListFilter?): List<Manga> {
val json = when (filter) {
is MangaListFilter.Search -> {
if (page > 1) {
return emptyList()
}
val url = "https://api.$domain/api/v1/search/serie"
val body = JSONObject()
body.put("title", filter.query.urlEncoded())
webClient.httpPost(url, body).parseJson().getJSONArray("data")
}
is MangaListFilter.Advanced -> {
val url = buildString {
append("https://api.")
append(domain)
append("/api/v1/")
if (filter.tags.isNotEmpty() || filter.states.isNotEmpty() || filter.contentRating.isNotEmpty()) {
if (page > 1) {
return emptyList()
}
append("search/advance?=")
if (filter.tags.isNotEmpty()) {
val tagQuery = filter.tags.joinToString(separator = ",") { it.key }
append("&genres=")
append(tagQuery)
}
if (filter.states.isNotEmpty()) {
filter.states.oneOrThrowIfMany()?.let {
append("&status=")
append(
when (it) {
MangaState.ONGOING -> "ongoing"
MangaState.FINISHED -> "completed"
MangaState.ABANDONED -> "droped"
MangaState.PAUSED -> "onhold"
MangaState.UPCOMING -> "soon"
},
)
}
}
filter.contentRating.oneOrThrowIfMany()?.let {
append("&adult=")
append(
when (it) {
ContentRating.ADULT -> "true"
else -> ""
},
)
}
append("&serie_type=webtoon")
} else {
append("webtoon/pages/latest/romance?page=")
append(page.toString())
}
}
webClient.httpGet(url).parseJson().getJSONArray("data")
}
null -> {
val url = "https://api.$domain/api/v1/webtoon/pages/latest/romance?page=$page"
webClient.httpGet(url).parseJson().getJSONArray("data")
}
}
return json.mapJSON { j ->
val href = "https://$domain/series/${j.getString("prefix")}-${j.getString("id")}-${j.getString("slug")}"
val cover = "https://api.$domain/storage/" + j.getString("thumbnail")
Manga(
id = generateUid(href),
title = j.getString("title"),
altTitle = null,
url = href,
publicUrl = href.toAbsoluteUrl(domain),
rating = RATING_UNKNOWN,
isNsfw = false,
coverUrl = cover,
tags = emptySet(),
state = when (j.getString("status")) {
"ongoing" -> MangaState.ONGOING
"completed" -> MangaState.FINISHED
"onhold" -> MangaState.PAUSED
"droped" -> MangaState.ABANDONED
else -> null
},
author = null,
source = source,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
val doc = webClient.httpGet("https://$domain/search/advance").parseHtml()
val json = JSONArray(doc.requireElementById("__NUXT_DATA__").data())
val tagsList = json.getJSONArray(3).toString().replace("[", "").replace("]", "").split(",")
return tagsList.mapNotNullToSet { idTag ->
val id = idTag.toInt()
val idKey = json.getJSONObject(id).getInt("id")
val key = json.getInt(idKey).toString()
val idName = json.getJSONObject(id).getInt("name")
val name = json.getString(idName)
MangaTag(
key = key,
title = name,
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val doc = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
val chaptersDeferred = async { loadChapters(manga.url) }
val json = JSONArray(doc.requireElementById("__NUXT_DATA__").data())
val descId = json.getJSONObject(6).getInt("story")
val desc = json.getString(descId)
val tagsId = json.getJSONObject(6).getInt("genres")
val tagsList = json.getJSONArray(tagsId).toString().replace("[", "").replace("]", "").split(",")
val ratingId = json.getJSONObject(6).getInt("rating")
val rating = json.getString(ratingId)
val nsfwId = json.getJSONObject(6).getInt("nsfw")
val nsfw = json.getBoolean(nsfwId)
manga.copy(
description = desc,
tags = tagsList.mapToSet { idTag ->
val id = idTag.toInt()
val idKey = json.getJSONObject(id).getInt("id")
val key = json.get(idKey).toString()
val idName = json.getJSONObject(id).getInt("name")
val name = json.get(idName).toString()
MangaTag(
key = key,
title = name,
source = source,
)
},
rating = rating?.toFloatOrNull()?.div(5f) ?: RATING_UNKNOWN,
isNsfw = nsfw,
chapters = chaptersDeferred.await(),
)
}
private val dateFormat = SimpleDateFormat("yyyy-MM-dd", sourceLocale)
private suspend fun loadChapters(baseUrl: String): List<MangaChapter> {
val key = baseUrl.substringAfter("-").substringBefore("-")
val seriesKey = baseUrl.substringAfterLast("/").substringBefore("-")
val json = JSONArray(webClient.httpGet("https://api.$domain/api/v1/webtoon/chapters/$key-desc").parseRaw())
return json.mapJSONIndexed { i, j ->
val url = "https://$domain/read/webtoon/$seriesKey-${j.getString("id")}-${j.getString("slug")}"
val date = j.getString("createdAt").substringBeforeLast("T")
MangaChapter(
id = generateUid(url),
url = url,
name = j.getString("slug").replace('-', ' '),
number = i + 1,
branch = null,
uploadDate = dateFormat.tryParse(date),
scanlator = null,
source = source,
)
}
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
val json = JSONArray(doc.requireElementById("__NUXT_DATA__").data())
val chapterData = json.getJSONObject(6).getInt("chapterData")
val pageLocate = json.getJSONObject(chapterData).getInt("webtoon")
val jsonPages = json.getJSONArray(pageLocate)
val pages = ArrayList<MangaPage>(jsonPages.length())
for (i in 0 until jsonPages.length()) {
val id = jsonPages.getInt(i)
val url = "https://api.$domain/storage/" + json.getString(id)
pages.add(
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
),
)
}
return pages
}
}

View File

@@ -0,0 +1,128 @@
package org.koitharu.kotatsu.parsers.site.ar
import okhttp3.Headers
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.network.UserAgents
import org.koitharu.kotatsu.parsers.util.*
import java.util.*
@MangaSourceParser("MANGASTORM", "MangaStorm", "ar")
internal class MangaStorm(context: MangaLoaderContext) : PagedMangaParser(context, MangaSource.MANGASTORM, 30) {
override val availableSortOrders: Set<SortOrder> = EnumSet.of(SortOrder.POPULARITY, SortOrder.UPDATED)
override val configKeyDomain = ConfigKey.Domain("mangastorm.org")
override val isMultipleTagsSupported = false
override val headers: Headers = Headers.Builder()
.add("User-Agent", UserAgents.CHROME_DESKTOP)
.build()
override suspend fun getListPage(page: Int, filter: MangaListFilter?): List<Manga> {
val url = buildString {
append("https://")
append(domain)
when (filter) {
is MangaListFilter.Search -> {
append("/mangas?page=")
append(page)
append("&query=")
append(filter.query.urlEncoded())
}
is MangaListFilter.Advanced -> {
if (filter.tags.isNotEmpty()) {
val tag = filter.tags.oneOrThrowIfMany()
append("/categories/")
append(tag?.key.orEmpty())
append("?page=")
append(page)
} else {
if (filter.sortOrder == SortOrder.POPULARITY) {
append("/mangas?page=")
append(page)
} else {
if (page > 1) {
return emptyList()
}
}
}
}
null -> {
append("/mangas?page=")
append(page)
}
}
}
val doc = webClient.httpGet(url).parseHtml()
return doc.select("div.row div.col").map { div ->
val href = div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
title = div.select(".manga-ct-title").text(),
altTitle = null,
url = href,
publicUrl = href.toAbsoluteUrl(domain),
rating = RATING_UNKNOWN,
isNsfw = false,
coverUrl = div.selectFirstOrThrow("img").src().orEmpty(),
tags = emptySet(),
state = null,
author = null,
source = source,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> = emptySet()
override suspend fun getDetails(manga: Manga): Manga {
val doc = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
val root = doc.selectFirstOrThrow(".card-body .col-lg-9")
return manga.copy(
altTitle = null,
state = null,
tags = root.select(".flex-wrap a").mapNotNullToSet { a ->
MangaTag(
key = a.attr("href").substringAfterLast('/'),
title = a.text(),
source = source,
)
},
author = null,
description = root.selectFirstOrThrow(".card-text").text(),
chapters = doc.select(".card-body a.btn-fixed-width").mapChapters(reversed = true) { i, a ->
val url = a.attrAsRelativeUrl("href")
MangaChapter(
id = generateUid(url),
name = a.text(),
number = i + 1,
url = url,
scanlator = null,
uploadDate = 0,
branch = null,
source = source,
)
},
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml().requireElementById("content")
return doc.select("div.text-center .img-fluid").map { img ->
val url = img.src()?.toRelativeUrl(domain) ?: img.parseFailed("Image src not found")
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
)
}
}
}

View File

@@ -0,0 +1,209 @@
package org.koitharu.kotatsu.parsers.site.ar
import kotlinx.coroutines.async
import kotlinx.coroutines.awaitAll
import kotlinx.coroutines.coroutineScope
import org.jsoup.nodes.Element
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("TEAMXNOVEL", "TeamXNovel", "ar")
internal class TeamXNovel(context: MangaLoaderContext) : PagedMangaParser(context, MangaSource.TEAMXNOVEL, 10) {
override val availableSortOrders: Set<SortOrder> = EnumSet.of(SortOrder.UPDATED, SortOrder.POPULARITY)
override val availableStates: Set<MangaState> =
EnumSet.of(MangaState.ONGOING, MangaState.FINISHED, MangaState.ABANDONED)
override val configKeyDomain = ConfigKey.Domain("team11x11.com")
override val isMultipleTagsSupported = false
override suspend fun getListPage(page: Int, filter: MangaListFilter?): List<Manga> {
val url = buildString {
append("https://")
append(domain)
when (filter) {
is MangaListFilter.Search -> {
append("/series?search=")
append(filter.query.urlEncoded())
if (page > 1) {
append("&page=")
append(page.toString())
}
}
is MangaListFilter.Advanced -> {
if (filter.tags.isNotEmpty()) {
val tag = filter.tags.oneOrThrowIfMany()
append("/series?genre=")
append(tag?.key.orEmpty())
if (page > 1) {
append("&page=")
append(page.toString())
}
append("&")
} else {
when (filter.sortOrder) {
SortOrder.POPULARITY -> append("/series")
SortOrder.UPDATED -> append("/")
else -> append("/")
}
if (page > 1) {
append("?page=")
append(page.toString())
append("&")
} else {
append("?")
}
}
if (filter.sortOrder == SortOrder.POPULARITY || filter.tags.isNotEmpty()) {
filter.states.oneOrThrowIfMany()?.let {
append("status=")
append(
when (it) {
MangaState.ONGOING -> "مستمرة"
MangaState.FINISHED -> "مكتمل"
MangaState.ABANDONED -> "متوقف"
else -> "مستمرة"
},
)
}
}
}
null -> append("/?page=$page")
}
}
val doc = webClient.httpGet(url).parseHtml()
return doc.select("div.listupd .bs .bsx").ifEmpty {
doc.select("div.post-body .box")
}.map { div ->
val href = div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
title = div.select(".tt, h3").text(),
altTitle = null,
url = href,
publicUrl = href.toAbsoluteUrl(domain),
rating = RATING_UNKNOWN,
isNsfw = false,
coverUrl = div.selectFirstOrThrow("img").src().orEmpty(),
tags = emptySet(),
state = when (div.selectFirst(".status")?.text()) {
"مستمرة" -> MangaState.ONGOING
"مكتمل" -> MangaState.FINISHED
"متوقف" -> MangaState.ABANDONED
else -> null
},
author = null,
source = source,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
val doc = webClient.httpGet("https://$domain/series").parseHtml()
return doc.requireElementById("select_genre").select("option").mapNotNullToSet {
MangaTag(
key = it.attr("value"),
title = it.text().toTitleCase(sourceLocale),
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga {
val doc = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
val mangaUrl = manga.url.toAbsoluteUrl(domain)
val maxPageChapterSelect = doc.select(".pagination .page-item a")
var maxPageChapter = 1
if (!maxPageChapterSelect.isNullOrEmpty()) {
maxPageChapterSelect.map {
val i = it.attr("href").substringAfterLast("=").toInt()
if (i > maxPageChapter) {
maxPageChapter = i
}
}
}
return manga.copy(
altTitle = null,
state = when (doc.selectFirstOrThrow(".full-list-info:contains(الحالة:) a").text()) {
"مستمرة" -> MangaState.ONGOING
"مكتمل" -> MangaState.FINISHED
"متوقف" -> MangaState.ABANDONED
else -> null
},
tags = doc.select(".review-author-info a").mapNotNullToSet { a ->
MangaTag(
key = a.attr("href").substringAfterLast("="),
title = a.text(),
source = source,
)
},
author = null,
description = doc.selectFirstOrThrow(".review-content").text(),
chapters = run {
if (maxPageChapter == 1) {
parseChapters(doc)
} else {
coroutineScope {
val result = ArrayList(parseChapters(doc))
result.ensureCapacity(result.size * maxPageChapter)
(2..maxPageChapter).map { i ->
async {
loadChapters(mangaUrl, i)
}
}.awaitAll()
.flattenTo(result)
result
}
}
}.reversed(),
)
}
private suspend fun loadChapters(baseUrl: String, page: Int): List<MangaChapter> {
return parseChapters(webClient.httpGet("$baseUrl?page=$page").parseHtml().body())
}
private val dateFormat = SimpleDateFormat("yyyy-MM-dd HH:mm:ss", sourceLocale)
private fun parseChapters(root: Element): List<MangaChapter> {
return root.requireElementById("chapter-contact").select(".eplister ul li")
.map { li ->
val url = li.selectFirstOrThrow("a").attrAsRelativeUrl("href")
MangaChapter(
id = generateUid(url),
name = li.selectFirstOrThrow(".epl-title").text(),
number = url.substringAfterLast('/').toIntOrNull() ?: 0,
url = url,
scanlator = null,
uploadDate = dateFormat.tryParse(li.selectFirstOrThrow(".epl-date").text()),
branch = null,
source = source,
)
}
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
return doc.select(".image_list img").map { img ->
val url = img.src()?.toRelativeUrl(domain) ?: img.parseFailed("Image src not found")
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
)
}
}
}

View File

@@ -1,4 +1,4 @@
package org.koitharu.kotatsu.parsers.site
package org.koitharu.kotatsu.parsers.site.be
import androidx.collection.ArraySet
import org.json.JSONArray
@@ -8,29 +8,30 @@ import org.koitharu.kotatsu.parsers.MangaParser
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.domain
import org.koitharu.kotatsu.parsers.util.generateUid
import org.koitharu.kotatsu.parsers.util.getDomain
import org.koitharu.kotatsu.parsers.util.json.mapJSON
import org.koitharu.kotatsu.parsers.util.json.mapJSONIndexed
import org.koitharu.kotatsu.parsers.util.json.mapJSONNotNull
import org.koitharu.kotatsu.parsers.util.json.stringIterator
import org.koitharu.kotatsu.parsers.util.toAbsoluteUrl
import java.util.*
@MangaSourceParser("ANIBEL", "Anibel", "be")
internal class AnibelParser(override val context: MangaLoaderContext) : MangaParser(MangaSource.ANIBEL) {
internal class AnibelParser(context: MangaLoaderContext) : MangaParser(context, MangaSource.ANIBEL) {
override val configKeyDomain = ConfigKey.Domain("anibel.net", null)
override val configKeyDomain = ConfigKey.Domain("anibel.net")
override val sortOrders: Set<SortOrder> = EnumSet.of(
override val availableSortOrders: Set<SortOrder> = EnumSet.of(
SortOrder.NEWEST,
)
override fun getFaviconUrl(): String {
return "https://cdn.${getDomain()}/favicons/favicon.png"
}
override suspend fun getList(
offset: Int,
query: String?,
tags: Set<MangaTag>?,
tagsExclude: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
if (!query.isNullOrEmpty()) {
@@ -73,12 +74,12 @@ internal class AnibelParser(override val context: MangaLoaderContext) : MangaPar
title = title.getString("be"),
coverUrl = jo.getString("poster").removePrefix("/cdn")
.toAbsoluteUrl(getDomain("cdn")) + "?width=200&height=280",
altTitle = title.getString("alt").takeUnless(String::isEmpty),
altTitle = title.optJSONArray("alt")?.optString(0)?.takeUnless(String::isEmpty),
author = null,
isNsfw = false,
rating = jo.getDouble("rating").toFloat() / 10f,
url = href,
publicUrl = "https://${getDomain()}/$href",
publicUrl = "https://${domain}/$href",
tags = jo.getJSONArray("genres").mapToTags(),
state = when (jo.getString("status")) {
"ongoing" -> MangaState.ONGOING
@@ -111,20 +112,21 @@ internal class AnibelParser(override val context: MangaLoaderContext) : MangaPar
""".trimIndent(),
).getJSONObject("media")
val title = details.getJSONObject("title")
val poster = details.getString("poster").removePrefix("/cdn")
.toAbsoluteUrl(getDomain("cdn"))
val poster = details.getString("poster").removePrefix("/cdn").toAbsoluteUrl(getDomain("cdn"))
val chapters = apiCall(
"""
chapters(mediaId: "${details.getString("mediaId")}") {
id
chapter
released
chapters(mediaId: "${details.getString("mediaId")}", offset: 0, limit: 99999) {
docs {
id
chapter
released
}
}
""".trimIndent(),
).getJSONArray("chapters")
).getJSONObject("chapters").getJSONArray("docs")
return manga.copy(
title = title.getString("be"),
altTitle = title.getString("alt"),
altTitle = title.optJSONArray("alt")?.optString(0)?.takeUnless(String::isEmpty),
coverUrl = "$poster?width=200&height=280",
largeCoverUrl = poster,
description = details.getJSONObject("description").getString("be"),
@@ -165,19 +167,17 @@ internal class AnibelParser(override val context: MangaLoaderContext) : MangaPar
""".trimIndent(),
).getJSONObject("chapter")
val pages = chapterJson.getJSONArray("images")
val chapterUrl = "https://${getDomain()}/${chapter.url}"
return pages.mapJSONIndexed { i, jo ->
MangaPage(
id = generateUid("${chapter.url}/$i"),
url = jo.getString("large"),
referer = chapterUrl,
preview = jo.getString("thumbnail"),
source = source,
)
}
}
override suspend fun getTags(): Set<MangaTag> {
override suspend fun getAvailableTags(): Set<MangaTag> {
val json = apiCall(
"""
getFilters(mediaType: manga) {
@@ -192,23 +192,29 @@ internal class AnibelParser(override val context: MangaLoaderContext) : MangaPar
private suspend fun search(query: String): List<Manga> {
val json = apiCall(
"""
search(query: "$query", limit: 40) {
id
search(query: "$query", limit: 60) {
mediaId
title {
be
en
}
poster
url
type
status
slug
mediaType
genres
}
""".trimIndent(),
)
val array = json.getJSONArray("search")
return array.mapJSON { jo ->
val mediaId = jo.getString("id")
return array.mapJSONNotNull { jo ->
val type = jo.getString("mediaType").lowercase()
if (type != "manga") {
return@mapJSONNotNull null
}
val mediaId = jo.getString("mediaId")
val title = jo.getJSONObject("title")
val href = "${jo.getString("type").lowercase()}/${jo.getString("url")}"
val href = "$type/${jo.getString("slug")}"
Manga(
id = generateUid(mediaId),
title = title.getString("be"),
@@ -219,17 +225,20 @@ internal class AnibelParser(override val context: MangaLoaderContext) : MangaPar
isNsfw = false,
rating = RATING_UNKNOWN,
url = href,
publicUrl = "https://${getDomain()}/$href",
tags = emptySet(),
state = null,
publicUrl = "https://${domain}/$href",
tags = jo.getJSONArray("genres").mapToTags(),
state = when (jo.getString("status")) {
"ongoing" -> MangaState.ONGOING
"finished" -> MangaState.FINISHED
else -> null
},
source = source,
)
}
}
private suspend fun apiCall(request: String): JSONObject {
return context.graphQLQuery("https://api.${getDomain()}/graphql", request)
.getJSONObject("data")
return webClient.graphQLQuery("https://${domain}/graphql", request).getJSONObject("data")
}
private fun JSONArray.mapToTags(): Set<MangaTag> {
@@ -242,6 +251,7 @@ internal class AnibelParser(override val context: MangaLoaderContext) : MangaPar
c == '-' -> {
builder.setCharAt(i, ' ')
}
capitalize -> {
builder.setCharAt(i, c.uppercaseChar())
capitalize = false
@@ -263,4 +273,4 @@ internal class AnibelParser(override val context: MangaLoaderContext) : MangaPar
}
return result
}
}
}

View File

@@ -0,0 +1,136 @@
package org.koitharu.kotatsu.parsers.site.en
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("BEETOON", "BeeToon.net", "en")
internal class BeeToon(context: MangaLoaderContext) :
PagedMangaParser(context, MangaSource.BEETOON, pageSize = 30) {
override val availableSortOrders: Set<SortOrder> = EnumSet.of(SortOrder.UPDATED, SortOrder.POPULARITY)
override val configKeyDomain = ConfigKey.Domain("manhwafull.net")
override val isMultipleTagsSupported = false
override suspend fun getListPage(page: Int, filter: MangaListFilter?): List<Manga> {
val url = buildString {
append("https://")
append(domain)
when (filter) {
is MangaListFilter.Search -> {
if (page > 1) {
return emptyList()
}
append("/?s=")
append(filter.query.urlEncoded())
}
is MangaListFilter.Advanced -> {
if (filter.tags.isNotEmpty()) {
val tag = filter.tags.oneOrThrowIfMany()
append("/genre/")
append(tag?.key.orEmpty())
append("/page-")
append(page)
append("/")
} else {
when (filter.sortOrder) {
SortOrder.UPDATED -> append("/latest-update/")
SortOrder.POPULARITY -> append("/popular-manga/")
else -> append("/latest-update/")
}
append("page-")
append(page)
append("/")
}
}
null -> append("/latest-update/page-$page/")
}
}
val doc = webClient.httpGet(url).parseHtml()
return doc.select(".comics-grid .entry").map { div ->
val href = div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(domain),
coverUrl = div.selectFirst("img")?.src().orEmpty(),
title = div.selectFirst(".name")?.text().orEmpty(),
altTitle = null,
rating = div.selectFirst(".counter")?.text()?.toFloatOrNull()?.div(5f) ?: RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = when (div.selectLastOrThrow(".status span").text()) {
"Ongoing" -> MangaState.ONGOING
"Completed" -> MangaState.FINISHED
else -> null
},
source = source,
isNsfw = isNsfwSource,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
val doc = webClient.httpGet("https://$domain/").parseHtml()
return doc.requireElementById("menu-item-3").select("ul.sub-menu li a").mapNotNullToSet {
MangaTag(
key = it.attr("href").removeSuffix('/').substringAfterLast('/'),
title = it.text(),
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga {
val doc = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
return manga.copy(
description = doc.getElementById("desc")?.text().orEmpty(),
rating = doc.selectFirst(".counter")?.text()?.toFloatOrNull()?.div(5f) ?: RATING_UNKNOWN,
tags = doc.body().select(".info .genre a").mapNotNullToSet {
MangaTag(
key = it.attr("href").removeSuffix('/').substringAfterLast('/'),
title = it.text(),
source = source,
)
},
author = doc.selectFirst(".info .author a")?.text(),
chapters = doc.select(".items-chapters a").mapChapters(reversed = true) { i, a ->
val url = a.attrAsRelativeUrl("href").toAbsoluteUrl(domain)
MangaChapter(
id = generateUid(url),
name = a.selectFirstOrThrow(".chap").text(),
number = i + 1,
url = url,
scanlator = null,
uploadDate = SimpleDateFormat("MM/dd/yyyy HH:mm:ss", Locale.ENGLISH)
.tryParse(a.selectFirst(".chapter-date")?.attr("title") ?: "0"),
branch = null,
source = source,
)
},
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val doc = webClient.httpGet(chapter.url.toAbsoluteUrl(domain)).parseHtml()
return doc.select(".chapter-content-inner center img").map { img ->
val urlPage = img.src()?.toRelativeUrl(domain) ?: img.parseFailed("Image src not found")
MangaPage(
id = generateUid(urlPage),
url = urlPage,
preview = null,
source = source,
)
}
}
}

View File

@@ -0,0 +1,107 @@
package org.koitharu.kotatsu.parsers.site.en
import org.koitharu.kotatsu.parsers.InternalParsersApi
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaParser
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.util.*
@MangaSourceParser("CLONEMANGA", "CloneManga", "en")
internal class CloneMangaParser(context: MangaLoaderContext) : MangaParser(context, MangaSource.CLONEMANGA) {
override val availableSortOrders: Set<SortOrder> = Collections.singleton(
SortOrder.POPULARITY,
)
override val configKeyDomain = ConfigKey.Domain("manga.clone-army.org")
@InternalParsersApi
override suspend fun getList(offset: Int, filter: MangaListFilter?): List<Manga> {
val link = when (filter) {
is MangaListFilter.Search -> {
return emptyList()
}
is MangaListFilter.Advanced -> {
if (offset > 0) {
return emptyList()
}
"https://$domain/viewer_landing.php"
}
null -> "https://$domain/viewer_landing.php"
}
val doc = webClient.httpGet(link).parseHtml()
val mangas = doc.getElementsByClass("comicPreviewContainer")
return mangas.mapNotNull { item ->
val background = item.selectFirstOrThrow(".comicPreview").styleValueOrNull("background")
val href = item.selectFirst("a")?.attrAsRelativeUrl("href") ?: return@mapNotNull null
val cover = background?.substring(background.indexOf("site/themes"), background.indexOf(")"))
Manga(
id = generateUid(href),
title = item.selectFirst("h3")?.text() ?: return@mapNotNull null,
coverUrl = "https://$domain/$cover",
altTitle = null,
author = "Dan Kim",
rating = RATING_UNKNOWN,
url = href,
isNsfw = false,
tags = emptySet(),
state = null,
publicUrl = href.toAbsoluteUrl(domain),
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga {
val doc = webClient.httpGet(manga.publicUrl).parseHtml()
val series = doc.location()
val numChapters = Regex(
pattern = "&page=(.*)&lang=",
).findAll(
input = doc.getElementsByTag("script")[3].toString(),
)
.elementAt(3).destructured.component1()
.toInt()
val chapters = ArrayList<MangaChapter>()
for (i in 0..numChapters) {
val chapter = MangaChapter(
id = generateUid("$series&page=$i"),
name = "Chapter ${i + 1}",
number = i + 1,
url = "$series&page=$i",
scanlator = null,
branch = null,
uploadDate = 0L,
source = source,
)
chapters.add(chapter)
}
return manga.copy(chapters = chapters)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val doc = webClient.httpGet(chapter.url.toAbsoluteUrl(domain)).parseHtml()
val imgUrl = doc.getElementsByClass("subsectionContainer")[0]
.selectFirst("img")
?.attrAsAbsoluteUrlOrNull("src") ?: doc.parseFailed("Something broken")
return listOf(
MangaPage(
id = generateUid(imgUrl),
url = imgUrl,
preview = null,
source = source,
),
)
}
override suspend fun getAvailableTags(): Set<MangaTag> = emptySet()
}

View File

@@ -0,0 +1,176 @@
package org.koitharu.kotatsu.parsers.site.en
import okhttp3.Headers
import org.koitharu.kotatsu.parsers.ErrorMessages
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.network.UserAgents
import org.koitharu.kotatsu.parsers.util.*
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("COMICEXTRA", "ComicExtra", "en")
internal class ComicExtra(context: MangaLoaderContext) : PagedMangaParser(context, MangaSource.COMICEXTRA, 25) {
override val availableSortOrders: Set<SortOrder> =
EnumSet.of(SortOrder.POPULARITY, SortOrder.UPDATED, SortOrder.NEWEST)
override val availableStates: Set<MangaState> = EnumSet.of(MangaState.ONGOING, MangaState.FINISHED)
override val configKeyDomain = ConfigKey.Domain("comicextra.me")
override val isMultipleTagsSupported = false
override val headers: Headers = Headers.Builder()
.add("User-Agent", UserAgents.CHROME_DESKTOP)
.build()
override suspend fun getListPage(page: Int, filter: MangaListFilter?): List<Manga> {
val url = buildString {
append("https://")
append(domain)
append("/")
when (filter) {
is MangaListFilter.Search -> {
append("comic-search?key=")
append(filter.query.urlEncoded())
if (page > 1) {
append("&page=")
append(page.toString())
}
}
is MangaListFilter.Advanced -> {
if (filter.tags.isNotEmpty() && filter.states.isEmpty()) {
filter.tags.oneOrThrowIfMany()?.let {
append(it.key)
}
} else if (filter.tags.isEmpty() && filter.states.isNotEmpty()) {
filter.states.oneOrThrowIfMany()?.let {
append(
when (it) {
MangaState.ONGOING -> "/ongoing-comic"
MangaState.FINISHED -> "/completed-comic"
else -> "/ongoing-comic"
},
)
}
} else if (filter.tags.isNotEmpty() && filter.states.isNotEmpty()) {
throw IllegalArgumentException(ErrorMessages.FILTER_BOTH_STATES_GENRES_NOT_SUPPORTED)
} else {
when (filter.sortOrder) {
SortOrder.POPULARITY -> append("popular-comic")
SortOrder.UPDATED -> append("new-comic")
SortOrder.NEWEST -> append("recent-comic")
else -> append("new-comic")
}
}
if (page > 1) {
append("/")
append(page.toString())
}
}
null -> {
append("popular-comic")
if (page > 1) {
append("/")
append(page.toString())
}
}
}
}
val doc = webClient.httpGet(url).parseHtml()
return doc.select("div.movie-list-index div.cartoon-box").map { div ->
val href = div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
title = div.selectFirstOrThrow("h3").text(),
altTitle = null,
url = href,
publicUrl = href.toAbsoluteUrl(domain),
rating = RATING_UNKNOWN,
isNsfw = false,
coverUrl = div.selectFirstOrThrow("img").attrAsAbsoluteUrl("src"),
tags = emptySet(),
state = when (div.selectFirstOrThrow(".detail:contains(Stasus: )").text()) {
"Stasus: Ongoing" -> MangaState.ONGOING
"Stasus: Completed" -> MangaState.FINISHED
else -> null
},
author = null,
source = source,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
val doc = webClient.httpGet("https://$domain/popular-comic").parseHtml()
return doc.select("li.tag-item a").mapNotNullToSet { a ->
MangaTag(
key = a.attr("href").substringAfterLast('/'),
title = a.text(),
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga {
val doc = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
val dateFormat = SimpleDateFormat("MM/dd/yy", sourceLocale)
return manga.copy(
altTitle = doc.selectFirstOrThrow("dt.movie-dt:contains(Alternate name:) + dd").text(),
state = when (doc.selectFirstOrThrow("dt.movie-dt:contains(Status:) + dd a").text()) {
"Ongoing" -> MangaState.ONGOING
"Completed" -> MangaState.FINISHED
else -> null
},
tags = doc.select("dt.movie-dt:contains(Genres:) + dd a").mapNotNullToSet { a ->
MangaTag(
key = a.attr("href").substringAfterLast("/"),
title = a.text(),
source = source,
)
},
author = doc.select("dt.movie-dt:contains(Author:) + dd").text(),
description = doc.getElementById("film-content")?.text(),
chapters = doc.requireElementById("list").select("tr")
.mapChapters(reversed = true) { i, tr ->
val a = tr.selectFirstOrThrow("a")
val url = a.attrAsRelativeUrl("href") + "/full"
val name = a.text()
val dateText = tr.select("td").last()?.text()
MangaChapter(
id = generateUid(url),
name = name,
number = i + 1,
url = url,
scanlator = null,
uploadDate = dateFormat.tryParse(dateText),
branch = null,
source = source,
)
},
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
return doc.select(".chapter-container img").map { img ->
val url = img.src()?.toRelativeUrl(domain) ?: img.parseFailed("Image src not found")
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
)
}
}
}

View File

@@ -0,0 +1,213 @@
package org.koitharu.kotatsu.parsers.site.en
import androidx.collection.ArraySet
import kotlinx.coroutines.async
import kotlinx.coroutines.awaitAll
import kotlinx.coroutines.coroutineScope
import okhttp3.Headers
import org.json.JSONArray
import org.jsoup.nodes.Document
import org.jsoup.nodes.Element
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.network.UserAgents
import org.koitharu.kotatsu.parsers.util.*
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("DYNASTYSCANS", "DynastyScans", "en")
internal class DynastyScans(context: MangaLoaderContext) : PagedMangaParser(context, MangaSource.DYNASTYSCANS, 117) {
override val availableSortOrders: Set<SortOrder> = EnumSet.of(SortOrder.ALPHABETICAL)
override val configKeyDomain = ConfigKey.Domain("dynasty-scans.com")
override val headers: Headers = Headers.Builder()
.add("User-Agent", UserAgents.CHROME_DESKTOP)
.build()
override val isMultipleTagsSupported = false
override suspend fun getListPage(page: Int, filter: MangaListFilter?): List<Manga> {
when (filter) {
is MangaListFilter.Search -> {
val url = buildString {
append("https://")
append(domain)
append("/search?q=")
append(filter.query.urlEncoded())
append("&")
append("classes[]".urlEncoded())
append("=Serie&page=")
append(page.toString())
}
return parseMangaListQuery(webClient.httpGet(url).parseHtml())
}
is MangaListFilter.Advanced -> {
val url = buildString {
append("https://")
append(domain)
if (filter.tags.isNotEmpty()) {
append("/tags/")
filter.tags.oneOrThrowIfMany()?.let {
append(it.key)
}
append("?view=groupings")
} else {
append("/series?view=cover")
}
append("&page=")
append(page.toString())
}
return parseMangaList(webClient.httpGet(url).parseHtml())
}
null -> {
val url = buildString {
append("https://")
append(domain)
append("/series?view=cover&page=")
append(page.toString())
}
return parseMangaList(webClient.httpGet(url).parseHtml())
}
}
}
private fun parseMangaList(doc: Document): List<Manga> {
return doc.select("li.span2")
.map { div ->
val href = div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
title = div.selectFirstOrThrow("div.caption").text(),
altTitle = null,
url = href,
publicUrl = href.toAbsoluteUrl(domain),
rating = RATING_UNKNOWN,
isNsfw = false,
coverUrl = div.selectFirstOrThrow("img").attrAsAbsoluteUrl("src"),
tags = setOf(),
state = null,
author = null,
source = source,
)
}
}
private fun parseMangaListQuery(doc: Document): List<Manga> {
return doc.select("dl.chapter-list dd")
.map { div ->
val href = div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
title = div.selectFirstOrThrow("a").text(),
altTitle = null,
url = href,
publicUrl = href.toAbsoluteUrl(domain),
rating = RATING_UNKNOWN,
isNsfw = false,
coverUrl = "",
tags = div.select("span.tags a").mapNotNullToSet { a ->
MangaTag(
key = a.attr("href").removeSuffix('/').substringAfterLast('/'),
title = a.text(),
source = source,
)
},
state = null,
author = null,
source = source,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
return coroutineScope {
(1..3).map { page ->
async { getTags(page) }
}
}.awaitAll().flattenTo(ArraySet(360))
}
private suspend fun getTags(page: Int): Set<MangaTag> {
val url = "https://$domain/tags?page=$page"
val root = webClient.httpGet(url).parseHtml()
return root.selectFirstOrThrow(".tag-list ").parseTags()
}
private fun Element.parseTags() = select("a").mapToSet {
MangaTag(
key = it.attr("href").removeSuffix('/').substringAfterLast('/'),
title = it.text(),
source = source,
)
}
override suspend fun getDetails(manga: Manga): Manga {
val doc = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
val chapters = getChapters(doc)
val root = doc.requireElementById("main")
return manga.copy(
altTitle = null,
state = when (root.select("h2.tag-title small").last()?.text()) {
"— Ongoing" -> MangaState.ONGOING
"— Completed", "— Completed and Licensed" -> MangaState.FINISHED
"— Dropped", "— Licensed and Removed", "— Abandoned" -> MangaState.ABANDONED
"— On Hiatus" -> MangaState.PAUSED
else -> null
},
coverUrl = root.selectFirst("img.thumbnail")?.src()
.orEmpty(), // It is needed if the manga was found via the search.
tags = root.selectFirstOrThrow("div.tag-tags").parseTags(),
author = null,
description = null,
chapters = chapters,
)
}
private fun getChapters(doc: Document): List<MangaChapter> {
val dateFormat = SimpleDateFormat("MMM dd yy", sourceLocale)
return doc.body().select("dl.chapter-list dd").mapChapters { i, li ->
val a = li.selectFirstOrThrow("a")
val href = a.attrAsRelativeUrl("href")
val dateText = li.select("small").last()?.text()?.replace("released ", "")?.replace("'", "")
MangaChapter(
id = generateUid(href),
name = a.text(),
number = i + 1,
url = href,
uploadDate = dateFormat.tryParse(dateText),
source = source,
scanlator = null,
branch = null,
)
}
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val chapterUrl = chapter.url.toAbsoluteUrl(domain)
val docs = webClient.httpGet(chapterUrl).parseHtml()
val script = docs.selectFirstOrThrow("script:containsData(var pages =)")
val json = JSONArray(script.data().substringAfter('=').substringBeforeLast(';'))
val pages = ArrayList<MangaPage>(json.length())
for (i in 0 until json.length()) {
val url = "https://" + domain + json.getString(i).substringAfter(":\"").substringBefore("\",")
pages.add(
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
),
)
}
return pages
}
}

View File

@@ -0,0 +1,199 @@
package org.koitharu.kotatsu.parsers.site.en
import kotlinx.coroutines.async
import kotlinx.coroutines.coroutineScope
import org.json.JSONArray
import org.koitharu.kotatsu.parsers.ErrorMessages
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import org.koitharu.kotatsu.parsers.util.json.mapJSON
import org.koitharu.kotatsu.parsers.util.json.toJSONList
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("FLIXSCANSORG", "FlixScans.org", "en")
internal class FlixScansOrg(context: MangaLoaderContext) : PagedMangaParser(context, MangaSource.FLIXSCANSORG, 18) {
override val availableSortOrders: Set<SortOrder> = EnumSet.of(SortOrder.UPDATED)
override val availableStates: Set<MangaState> = EnumSet.allOf(MangaState::class.java)
override val availableContentRating: Set<ContentRating> = EnumSet.of(ContentRating.ADULT)
override val configKeyDomain = ConfigKey.Domain("flixscans.org")
override val isSearchSupported = false
override suspend fun getListPage(page: Int, filter: MangaListFilter?): List<Manga> {
val json = when (filter) {
is MangaListFilter.Search -> {
throw IllegalArgumentException(ErrorMessages.SEARCH_NOT_SUPPORTED)
}
is MangaListFilter.Advanced -> {
val url = buildString {
append("https://api.")
append(domain)
append("/api/v1/search/advance?=&serie_type=webtoon&page=")
append(page.toString())
append("&genres=")
append(filter.tags.joinToString(separator = ",") { it.key })
filter.states.oneOrThrowIfMany()?.let {
append("&status=")
append(
when (it) {
MangaState.ONGOING -> "ongoing"
MangaState.FINISHED -> "completed"
MangaState.ABANDONED -> "droped"
MangaState.PAUSED -> "onhold"
MangaState.UPCOMING -> "soon"
},
)
}
filter.contentRating.oneOrThrowIfMany()?.let {
append("&adult=")
append(
when (it) {
ContentRating.ADULT -> "true"
else -> ""
},
)
}
append("&serie_type=webtoon")
}
webClient.httpGet(url).parseJson().getJSONArray("data")
}
null -> {
val url = "https://api.$domain/api/v1/search/advance?=&serie_type=webtoon&page=$page"
webClient.httpGet(url).parseJson().getJSONArray("data")
}
}
return json.mapJSON { j ->
val href = "https://$domain/series/${j.getString("prefix")}-${j.getString("id")}-${j.getString("slug")}"
val cover = "https://media.$domain/" + j.getString("thumbnail")
Manga(
id = generateUid(href),
title = j.getString("title"),
altTitle = null,
url = href,
publicUrl = href.toAbsoluteUrl(domain),
rating = RATING_UNKNOWN,
isNsfw = isNsfwSource,
coverUrl = cover,
tags = emptySet(),
state = when (j.getString("status")) {
"ongoing" -> MangaState.ONGOING
"completed" -> MangaState.FINISHED
"onhold" -> MangaState.PAUSED
"droped" -> MangaState.ABANDONED
else -> null
},
author = null,
source = source,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
val doc = webClient.httpGet("https://$domain/search/advance").parseHtml()
val json = JSONArray(doc.requireElementById("__NUXT_DATA__").data())
val tagsList = json.getJSONArray(3).toString().replace("[", "").replace("]", "").split(",")
return tagsList.mapNotNullToSet { idTag ->
val id = idTag.toInt()
val idKey = json.getJSONObject(id).getInt("id")
val key = json.getInt(idKey).toString()
val idName = json.getJSONObject(id).getInt("name")
val name = json.getString(idName)
MangaTag(
key = key,
title = name,
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val doc = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
val chaptersDeferred = async { loadChapters(manga.url) }
val json = JSONArray(doc.requireElementById("__NUXT_DATA__").data())
val descId = json.getJSONObject(6).getInt("story")
val desc = json.getString(descId)
val tagsId = json.getJSONObject(6).getInt("genres")
val tagsList = json.getJSONArray(tagsId).toString().replace("[", "").replace("]", "").split(",")
val ratingId = json.getJSONObject(6).getInt("rating")
val rating = json.getString(ratingId)
val nsfwId = json.getJSONObject(6).getInt("nsfw")
val nsfw = json.getBoolean(nsfwId)
manga.copy(
description = desc,
tags = tagsList.mapToSet { idTag ->
val id = idTag.toInt()
val idKey = json.getJSONObject(id).getInt("id")
val key = json.get(idKey).toString()
val idName = json.getJSONObject(id).getInt("name")
val name = json.get(idName).toString()
MangaTag(
key = key,
title = name,
source = source,
)
},
rating = rating?.toFloatOrNull()?.div(5f) ?: RATING_UNKNOWN,
isNsfw = nsfw,
chapters = chaptersDeferred.await(),
)
}
private val dateFormat = SimpleDateFormat("yyyy-MM-dd", sourceLocale)
private suspend fun loadChapters(baseUrl: String): List<MangaChapter> {
val key = baseUrl.substringAfter("-").substringBefore("-")
val seriesKey = baseUrl.substringAfterLast("/").substringBefore("-")
val json = JSONArray(
webClient.httpGet("https://api.$domain/api/v1/webtoon/chapters/$key-desc").parseRaw(),
).toJSONList().reversed()
return json.mapChapters { i, j ->
val url = "https://$domain/read/webtoon/$seriesKey-${j.getString("id")}-${j.getString("slug")}"
val date = j.getString("createdAt").substringBeforeLast("T")
MangaChapter(
id = generateUid(url),
url = url,
name = j.getString("slug").replace('-', ' '),
number = i + 1,
branch = null,
uploadDate = dateFormat.tryParse(date),
scanlator = null,
source = source,
)
}
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
val json = JSONArray(doc.requireElementById("__NUXT_DATA__").data())
val chapterData = json.getJSONObject(6).getInt("chapterData")
val pageLocate = json.getJSONObject(chapterData).getInt("webtoon")
val jsonPages = json.getJSONArray(pageLocate)
val pages = ArrayList<MangaPage>(jsonPages.length())
for (i in 0 until jsonPages.length()) {
val id = jsonPages.getInt(i)
val url = "https://media.$domain/" + json.getString(id)
pages.add(
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
),
)
}
return pages
}
}

View File

@@ -0,0 +1,164 @@
package org.koitharu.kotatsu.parsers.site.en
import kotlinx.coroutines.async
import kotlinx.coroutines.coroutineScope
import okhttp3.Headers
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.network.UserAgents
import org.koitharu.kotatsu.parsers.util.*
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("MANGAGEKO", "MangaGeko", "en")
internal class MangaGeko(context: MangaLoaderContext) : PagedMangaParser(context, MangaSource.MANGAGEKO, 30) {
override val availableSortOrders: Set<SortOrder> =
EnumSet.of(SortOrder.POPULARITY, SortOrder.UPDATED, SortOrder.NEWEST)
override val configKeyDomain = ConfigKey.Domain("www.mgeko.com", "www.mangageko.com")
override val isMultipleTagsSupported = false
override val headers: Headers = Headers.Builder()
.add("User-Agent", UserAgents.CHROME_DESKTOP)
.build()
override suspend fun getListPage(page: Int, filter: MangaListFilter?): List<Manga> {
val url = buildString {
append("https://")
append(domain)
when (filter) {
is MangaListFilter.Search -> {
if (page > 1) {
return emptyList()
}
append("/search/?search=")
append(filter.query.urlEncoded())
}
is MangaListFilter.Advanced -> {
append("/browse-comics/?results=")
append(page)
append("&filter=")
when (filter.sortOrder) {
SortOrder.POPULARITY -> append("views")
SortOrder.UPDATED -> append("Updated")
SortOrder.NEWEST -> append("New")
else -> append("Updated")
}
if (filter.tags.isNotEmpty()) {
filter.tags.oneOrThrowIfMany()?.let {
append("&genre=")
append(it.key)
}
}
}
null -> {
append("/browse-comics/?results=")
append(page)
append("&filter=Updated")
}
}
}
val doc = webClient.httpGet(url).parseHtml()
return doc.select("li.novel-item").map { div ->
val href = div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
title = div.selectFirstOrThrow("h4").text(),
altTitle = null,
url = href,
publicUrl = href.toAbsoluteUrl(domain),
rating = RATING_UNKNOWN,
isNsfw = false,
coverUrl = div.selectFirstOrThrow("img").src().orEmpty(),
tags = emptySet(),
state = null,
author = div.selectFirstOrThrow("h6").text().removePrefix("Author(S): "),
source = source,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
val doc = webClient.httpGet("https://$domain/browse-comics/").parseHtml()
return doc.select("label.checkbox-inline").mapNotNullToSet { label ->
MangaTag(
key = label.selectFirstOrThrow("input").attr("value"),
title = label.text(),
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val doc = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
val chaptersDeferred = async { loadChapters(manga.url) }
manga.copy(
altTitle = doc.selectFirstOrThrow(".alternative-title").text(),
state = when (doc.selectFirstOrThrow(".header-stats span:contains(Status) strong").text()) {
"Ongoing" -> MangaState.ONGOING
"Completed" -> MangaState.FINISHED
else -> null
},
tags = doc.select(".categories ul li a").mapNotNullToSet { a ->
MangaTag(
key = a.attr("href").substringAfterLast("="),
title = a.text(),
source = source,
)
},
author = doc.selectFirstOrThrow(".author").text(),
description = doc.selectFirstOrThrow(".description").html(),
chapters = chaptersDeferred.await(),
)
}
private suspend fun loadChapters(mangaUrl: String): List<MangaChapter> {
val urlChapter = mangaUrl + "all-chapters/"
val doc = webClient.httpGet(urlChapter.toAbsoluteUrl(domain)).parseHtml()
val dateFormat = SimpleDateFormat("MMM dd, yyyy", sourceLocale)
return doc.requireElementById("chapters").select("ul.chapter-list li")
.mapChapters(reversed = true) { i, li ->
val a = li.selectFirstOrThrow("a")
val url = a.attrAsRelativeUrl("href")
val name = li.selectFirstOrThrow(".chapter-title").text()
val dateText = li.select(".chapter-update").attr("datetime").substringBeforeLast(',')
.replace(".", "").replace("Sept", "Sep")
MangaChapter(
id = generateUid(url),
name = name,
number = i + 1,
url = url,
scanlator = null,
uploadDate = dateFormat.tryParse(dateText),
branch = null,
source = source,
)
}
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
return doc.requireElementById("chapter-reader").select("img").map { img ->
val url = img.src()?.toRelativeUrl(domain) ?: img.parseFailed("Image src not found")
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
)
}
}
}

View File

@@ -1,4 +1,4 @@
package org.koitharu.kotatsu.parsers.site
package org.koitharu.kotatsu.parsers.site.en
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaParser
@@ -11,49 +11,83 @@ import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("MANGATOWN", "MangaTown", "en")
internal class MangaTownParser(override val context: MangaLoaderContext) : MangaParser(MangaSource.MANGATOWN) {
internal class MangaTownParser(context: MangaLoaderContext) : MangaParser(context, MangaSource.MANGATOWN) {
override val configKeyDomain = ConfigKey.Domain("www.mangatown.com", null)
override val configKeyDomain = ConfigKey.Domain("www.mangatown.com")
override val sortOrders: Set<SortOrder> = EnumSet.of(
override val availableSortOrders: Set<SortOrder> = EnumSet.of(
SortOrder.ALPHABETICAL,
SortOrder.RATING,
SortOrder.POPULARITY,
SortOrder.UPDATED,
)
private val regexTag = Regex("[^\\-]+-[^\\-]+-[^\\-]+-[^\\-]+-[^\\-]+-[^\\-]+")
override val availableStates: Set<MangaState> = EnumSet.of(
MangaState.ONGOING,
MangaState.FINISHED,
)
override suspend fun getList(
offset: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val sortKey = when (sortOrder) {
SortOrder.ALPHABETICAL -> "?name.az"
SortOrder.RATING -> "?rating.za"
SortOrder.UPDATED -> "?last_chapter_time.za"
else -> ""
}
override val isMultipleTagsSupported = false
override suspend fun getList(offset: Int, filter: MangaListFilter?): List<Manga> {
val page = (offset / 30) + 1
val url = when {
!query.isNullOrEmpty() -> {
if (offset != 0) {
return emptyList()
val url = buildString {
append("https://")
append(domain)
when (filter) {
is MangaListFilter.Search -> {
append("/search?name=")
append(filter.query.urlEncoded())
append("&page=")
append(page.toString())
}
"/search?name=${query.urlEncoded()}".toAbsoluteUrl(getDomain())
}
tags.isNullOrEmpty() -> "/directory/$page.htm$sortKey".toAbsoluteUrl(getDomain())
tags.size == 1 -> "/directory/${tags.first().key}/$page.htm$sortKey".toAbsoluteUrl(getDomain())
else -> tags.joinToString(
prefix = "/search?page=$page".toAbsoluteUrl(getDomain()),
) { tag ->
"&genres[${tag.key}]=1"
is MangaListFilter.Advanced -> {
append("/directory/")
append("0-")
if (filter.tags.isNotEmpty()) {
filter.tags.oneOrThrowIfMany()?.let {
append(it.key)
}
} else {
append("0")
}
append("-0-")
if (filter.states.isNotEmpty()) {
filter.states.oneOrThrowIfMany()?.let {
append(
when (it) {
MangaState.ONGOING -> "ongoing"
MangaState.FINISHED -> "completed"
else -> "0"
},
)
}
} else {
append("0")
}
append("-0-0/")
append(page.toString())
append(".htm")
append(
when (filter.sortOrder) {
SortOrder.POPULARITY -> ""
SortOrder.UPDATED -> "?last_chapter_time.za"
SortOrder.ALPHABETICAL -> "?name.az"
SortOrder.RATING -> "?rating.za"
else -> "?last_chapter_time.za"
},
)
}
null -> append("/directory/$page.htm?last_chapter_time.za")
}
}
val doc = context.httpGet(url).parseHtml()
val doc = webClient.httpGet(url).parseHtml()
val root = doc.body().selectFirstOrThrow("ul.manga_pic_list")
return root.select("li").mapNotNull { li ->
val a = li.selectFirst("a.manga_cover")
@@ -81,19 +115,19 @@ internal class MangaTownParser(override val context: MangaLoaderContext) : Manga
tags = li.selectFirst("p.keyWord")?.select("a")?.mapNotNullToSet tags@{ x ->
MangaTag(
title = x.attr("title").toTitleCase(),
key = x.attr("href").parseTagKey() ?: return@tags null,
key = x.attr("href").substringAfter("/directory/0-").substringBefore("-0-"),
source = MangaSource.MANGATOWN,
)
}.orEmpty(),
url = href,
isNsfw = false,
publicUrl = href.toAbsoluteUrl(a.host ?: getDomain()),
publicUrl = href.toAbsoluteUrl(a.host ?: domain),
)
}
}
override suspend fun getDetails(manga: Manga): Manga {
val doc = context.httpGet(manga.url.toAbsoluteUrl(getDomain())).parseHtml()
val doc = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
val root = doc.body().selectFirstOrThrow("section.main")
.selectFirstOrThrow("div.article_content")
val info = root.selectFirst("div.detail_info")?.selectFirst("ul")
@@ -106,7 +140,7 @@ internal class MangaTownParser(override val context: MangaLoaderContext) : Manga
}?.select("a")?.mapNotNull { a ->
MangaTag(
title = a.attr("title").toTitleCase(),
key = a.attr("href").parseTagKey() ?: return@mapNotNull null,
key = a.attr("href").substringAfter("/directory/0-").substringBefore("-0-"),
source = MangaSource.MANGATOWN,
)
}.orEmpty(),
@@ -135,8 +169,8 @@ internal class MangaTownParser(override val context: MangaLoaderContext) : Manga
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(getDomain())
val doc = context.httpGet(fullUrl).parseHtml()
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
val root = doc.body().selectFirstOrThrow("div.page_select")
return root.selectFirstOrThrow("select").selectOrThrow("option").mapNotNull {
val href = it.attrAsRelativeUrlOrNull("value")
@@ -147,29 +181,25 @@ internal class MangaTownParser(override val context: MangaLoaderContext) : Manga
id = generateUid(href),
url = href,
preview = null,
referer = fullUrl,
source = MangaSource.MANGATOWN,
)
}
}
override suspend fun getPageUrl(page: MangaPage): String {
val doc = context.httpGet(page.url.toAbsoluteUrl(getDomain())).parseHtml()
val doc = webClient.httpGet(page.url.toAbsoluteUrl(domain)).parseHtml()
return doc.requireElementById("image").attrAsAbsoluteUrl("src")
}
override suspend fun getTags(): Set<MangaTag> {
val doc = context.httpGet("/directory/".toAbsoluteUrl(getDomain())).parseHtml()
override suspend fun getAvailableTags(): Set<MangaTag> {
val doc = webClient.httpGet("/directory/".toAbsoluteUrl(domain)).parseHtml()
val root = doc.body().selectFirst("aside.right")
?.getElementsContainingOwnText("Genres")
?.first()
?.nextElementSibling() ?: doc.parseFailed("Root not found")
return root.select("li").mapNotNullToSet { li ->
val a = li.selectFirst("a") ?: return@mapNotNullToSet null
val key = a.attr("href").parseTagKey()
if (key.isNullOrEmpty()) {
return@mapNotNullToSet null
}
val key = a.attr("href").substringAfter("/directory/0-").substringBefore("-0-")
MangaTag(
source = MangaSource.MANGATOWN,
key = key,
@@ -188,7 +218,7 @@ internal class MangaTownParser(override val context: MangaLoaderContext) : Manga
}
private suspend fun bypassLicensedChapters(manga: Manga): List<MangaChapter> {
val doc = context.httpGet(manga.url.toAbsoluteUrl(getDomain("m"))).parseHtml()
val doc = webClient.httpGet(manga.url.toAbsoluteUrl(getDomain("m"))).parseHtml()
val list = doc.body().selectFirst("ul.detail-ch-list") ?: return emptyList()
val dateFormat = SimpleDateFormat("MMM dd,yyyy", Locale.US)
return list.select("li").asReversed().mapIndexedNotNull { i, li ->
@@ -212,6 +242,4 @@ internal class MangaTownParser(override val context: MangaLoaderContext) : Manga
)
}
}
private fun String.parseTagKey() = split('/').findLast { regexTag matches it }
}
}

View File

@@ -0,0 +1,183 @@
package org.koitharu.kotatsu.parsers.site.en
import kotlinx.coroutines.coroutineScope
import okhttp3.Headers
import org.jsoup.nodes.Document
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.network.UserAgents
import org.koitharu.kotatsu.parsers.util.*
import org.koitharu.kotatsu.parsers.util.json.mapJSON
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("MANGAOWL", "MangaOwl.to", "en")
internal class Mangaowl(context: MangaLoaderContext) :
PagedMangaParser(context, MangaSource.MANGAOWL, pageSize = 24) {
override val availableSortOrders: Set<SortOrder> = EnumSet.of(
SortOrder.POPULARITY,
SortOrder.NEWEST,
SortOrder.UPDATED,
SortOrder.RATING,
)
override val availableStates: Set<MangaState> = EnumSet.of(MangaState.ONGOING, MangaState.FINISHED)
override val configKeyDomain = ConfigKey.Domain("mangaowl.to")
override val headers: Headers = Headers.Builder()
.add("User-Agent", UserAgents.CHROME_DESKTOP)
.build()
override suspend fun getListPage(page: Int, filter: MangaListFilter?): List<Manga> {
val url = buildString {
append("https://")
append(domain)
when (filter) {
is MangaListFilter.Search -> {
append("/10-search?q=")
append(filter.query.urlEncoded())
append("&page=")
append(page.toString())
}
is MangaListFilter.Advanced -> {
append("/10-comics")
append("?page=")
append(page.toString())
filter.tags.forEach { tag ->
append("&genres=")
append(tag.key)
}
filter.states.oneOrThrowIfMany()?.let {
append("&status=")
append(
when (it) {
MangaState.ONGOING -> "ongoing"
MangaState.FINISHED -> "completed"
else -> ""
},
)
}
append("&ordering=")
append(
when (filter.sortOrder) {
SortOrder.POPULARITY -> "view_count"
SortOrder.UPDATED -> "-modified_at"
SortOrder.NEWEST -> "created_at"
SortOrder.RATING -> "rating"
else -> "modified_at"
},
)
}
null -> {
append("/10-comics?ordering=-modified_at&page=")
append(page.toString())
}
}
}
val doc = webClient.httpGet(url).parseHtml()
return doc.select("div.manga-item.column").map { div ->
val href = div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(div.host ?: domain),
coverUrl = div.selectFirst("img")?.src().orEmpty(),
title = div.selectFirst("a.one-line")?.text().orEmpty(),
altTitle = null,
rating = div.select("span").last()?.text()?.toFloatOrNull()?.div(5f) ?: RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = null,
source = source,
isNsfw = false,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
val doc = webClient.httpGet("https://$domain/10-genres").parseHtml()
return doc.select("div.genres-container span.genre-item a").mapNotNullToSet { a ->
val key = a.attr("href").removeSuffix('/').substringAfterLast('/').substringBefore("-")
MangaTag(
key = key,
title = a.text(),
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val fullUrl = manga.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
manga.copy(
tags = doc.body().select("div.comic-attrs div.column.my-2:contains(Genres) a").mapNotNullToSet { a ->
MangaTag(
key = a.attr("href").removeSuffix("/").substringAfterLast('/').substringBefore("-"),
title = a.text().toTitleCase().replace(",", ""),
source = source,
)
},
description = doc.select("span.story-desc").html(),
state = when (doc.select("div.section-status:contains(Status) span").last()?.text()) {
"Ongoing" -> MangaState.ONGOING
"Completed" -> MangaState.FINISHED
else -> null
},
chapters = getChapters(manga.url, doc),
)
}
private fun getChapters(mangaUrl: String, doc: Document): List<MangaChapter> {
val dateFormat = SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSSSSS'Z'", sourceLocale)
val script = doc.selectFirstOrThrow("script:containsData(chapters:)")
val json = script.data().substringAfter("chapters:[")
.substringBeforeLast(')')
.substringBefore("],latest_chapter:")
.split("},")
val slug = mangaUrl.substringAfterLast("/")
return json.mapChapters { i, t ->
if (t.contains("Chapter")) {
val id = t.substringAfter("id:").substringBefore(",created_at")
val url = "/reading/$slug/$id"
val date = t.substringAfter("created_at:\"").substringBefore('"')
val name = t.substringAfter("name:\"").substringBefore('"')
MangaChapter(
id = generateUid(url),
name = name,
number = i + 1,
url = url,
uploadDate = dateFormat.tryParse(date),
source = source,
scanlator = null,
branch = null,
)
} else {
null
}
}
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val id = chapter.url.substringAfterLast("/")
val json = webClient.httpGet("https://api.mangaowl.to/v1/chapters/$id/images?page_size=100").parseJson()
return json.getJSONArray("results").mapJSON { jo ->
MangaPage(
id = generateUid(jo.getString("image")),
preview = null,
source = chapter.source,
url = jo.getString("image"),
)
}
}
}

View File

@@ -0,0 +1,239 @@
package org.koitharu.kotatsu.parsers.site.en
import androidx.collection.ArrayMap
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.util.*
@MangaSourceParser("MANHWA18", "Manhwa18.net", "en", type = ContentType.HENTAI)
class Manhwa18Parser(context: MangaLoaderContext) :
PagedMangaParser(context, MangaSource.MANHWA18, pageSize = 18, searchPageSize = 18) {
override val configKeyDomain: ConfigKey.Domain = ConfigKey.Domain("manhwa18.net")
override val availableSortOrders: Set<SortOrder>
get() = EnumSet.of(
SortOrder.UPDATED,
SortOrder.POPULARITY,
SortOrder.ALPHABETICAL,
SortOrder.NEWEST,
SortOrder.RATING,
)
override val availableStates: Set<MangaState> = EnumSet.of(
MangaState.ONGOING,
MangaState.FINISHED,
MangaState.PAUSED,
)
override val isTagsExclusionSupported = true
private val tagsMap = SuspendLazy(::parseTags)
override suspend fun getFavicons(): Favicons {
return Favicons(
listOf(
Favicon("https://$domain/uploads/logos/logo-mini.png", 92, null),
),
domain,
)
}
override suspend fun getListPage(page: Int, filter: MangaListFilter?): List<Manga> {
val url = buildString {
append("https://")
append(domain)
append("/tim-kiem?page=")
append(page.toString())
when (filter) {
is MangaListFilter.Search -> {
append("&q=")
append(filter.query.urlEncoded())
}
is MangaListFilter.Advanced -> {
append("&accept_genres=")
if (filter.tags.isNotEmpty()) {
append(
filter.tags.joinToString(",") { it.key },
)
}
append("&reject_genres=")
if (filter.tagsExclude.isNotEmpty()) {
append(
filter.tagsExclude.joinToString(",") { it.key },
)
}
append("&sort=")
append(
when (filter.sortOrder) {
SortOrder.ALPHABETICAL -> "az"
SortOrder.ALPHABETICAL_DESC -> "za"
SortOrder.POPULARITY -> "top"
SortOrder.UPDATED -> "update"
SortOrder.NEWEST -> "new"
SortOrder.RATING -> "like"
},
)
filter.states.oneOrThrowIfMany()?.let {
append("&status=")
append(
when (it) {
MangaState.ONGOING -> "1"
MangaState.FINISHED -> "3"
MangaState.PAUSED -> "2"
else -> ""
},
)
}
}
null -> append("&sort=update")
}
}
val docs = webClient.httpGet(url).parseHtml()
return docs.select(".card-body .thumb-item-flow")
.map {
val titleElement = it.selectFirstOrThrow(".thumb_attr.series-title > a")
val absUrl = titleElement.attrAsAbsoluteUrl("href")
Manga(
id = generateUid(absUrl.toRelativeUrl(domain)),
title = titleElement.text(),
altTitle = null,
url = absUrl.toRelativeUrl(domain),
publicUrl = absUrl,
rating = RATING_UNKNOWN,
isNsfw = true,
coverUrl = it.selectFirst("div.img-in-ratio")?.attrAsAbsoluteUrl("data-bg").orEmpty(),
tags = emptySet(),
state = null,
author = null,
largeCoverUrl = null,
description = null,
source = MangaSource.MANHWA18,
)
}
}
override suspend fun getDetails(manga: Manga): Manga {
val docs = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
val cardInfoElement = docs.selectFirst("div.series-information")
val author = cardInfoElement?.selectFirst(".info-name:contains(Author)")?.parent()
?.select("a")
?.joinToString(", ") { it.text() }
val availableTags = tagsMap.get()
val tags = cardInfoElement?.selectFirst(".info-name:contains(Genre)")?.parent()
?.select("a")
?.mapNotNullToSet { availableTags[it.text().lowercase(Locale.ENGLISH)] }
val state = cardInfoElement?.selectFirst(".info-name:contains(Status)")?.parent()
?.selectFirst("a")
?.let {
when (it.text().lowercase()) {
"on going" -> MangaState.ONGOING
"completed" -> MangaState.FINISHED
"on hold" -> MangaState.PAUSED
else -> null
}
}
return manga.copy(
altTitle = cardInfoElement?.selectFirst("b:contains(Other names)")?.parent()?.ownText()?.removePrefix(": "),
author = author,
description = docs.selectFirst(".series-summary .summary-content")?.html(),
tags = tags.orEmpty(),
state = state,
chapters = docs.select(".card-body > .list-chapters > a").mapChapters(reversed = true) { index, element ->
val chapterUrl = element.attrAsAbsoluteUrlOrNull("href")?.toRelativeUrl(domain)
?: return@mapChapters null
val uploadDate = parseUploadDate(element.selectFirst(".chapter-time")?.text())
MangaChapter(
id = generateUid(chapterUrl),
name = element.selectFirst(".chapter-name")?.text().orEmpty(),
number = index + 1,
url = chapterUrl,
scanlator = null,
uploadDate = uploadDate,
branch = null,
source = MangaSource.MANHWA18,
)
},
)
}
// 7 minutes ago
// 5 hours ago
// 2 days ago
// 2 weeks ago
// 4 years ago
private fun parseUploadDate(timeStr: String?): Long {
timeStr ?: return 0
val timeWords = timeStr.split(' ')
if (timeWords.size != 3) return 0
val timeWord = timeWords[1]
val timeAmount = timeWords[0].toIntOrNull() ?: return 0
val timeUnit = when (timeWord) {
"minute", "minutes" -> Calendar.MINUTE
"hour", "hours" -> Calendar.HOUR
"day", "days" -> Calendar.DAY_OF_YEAR
"week", "weeks" -> Calendar.WEEK_OF_YEAR
"month", "months" -> Calendar.MONTH
"year", "years" -> Calendar.YEAR
else -> return 0
}
val cal = Calendar.getInstance()
cal.add(timeUnit, -timeAmount)
return cal.time.time
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val chapterUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(chapterUrl).parseHtml()
return doc.requireElementById("chapter-content").select("img").mapNotNull {
val url = it.attrAsRelativeUrlOrNull("data-src")
?: it.attrAsRelativeUrlOrNull("src")
?: return@mapNotNull null
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = MangaSource.MANHWA18,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
return tagsMap.get().values.toSet()
}
private suspend fun parseTags(): Map<String, MangaTag> {
val doc = webClient.httpGet("https://$domain/tim-kiem?q=").parseHtml()
val list = doc.getElementsByAttribute("data-genre-id")
if (list.isEmpty()) {
return emptyMap()
}
val result = ArrayMap<String, MangaTag>(list.size)
for (item in list) {
val id = item.attr("data-genre-id")
val name = item.text()
result[name.lowercase(Locale.ENGLISH)] = MangaTag(
title = name.toTitleCase(Locale.ENGLISH),
key = id,
source = source,
)
}
return result
}
}

View File

@@ -0,0 +1,165 @@
package org.koitharu.kotatsu.parsers.site.en
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.text.DateFormat
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("MANHWASMEN", "ManhwasMen", "en", type = ContentType.HENTAI)
class ManhwasMen(context: MangaLoaderContext) :
PagedMangaParser(context, MangaSource.MANHWASMEN, pageSize = 30, searchPageSize = 30) {
override val configKeyDomain: ConfigKey.Domain = ConfigKey.Domain("manhwas.men")
override val isMultipleTagsSupported = false
override val availableSortOrders: Set<SortOrder>
get() = EnumSet.of(SortOrder.POPULARITY)
override val availableStates: Set<MangaState> = EnumSet.of(MangaState.ONGOING, MangaState.FINISHED)
override suspend fun getListPage(page: Int, filter: MangaListFilter?): List<Manga> {
val url = buildString {
append("https://")
append(domain)
append("/manga-list")
append("?page=")
append(page.toString())
when (filter) {
is MangaListFilter.Search -> {
append("&search=")
append(filter.query.urlEncoded())
}
is MangaListFilter.Advanced -> {
filter.tags.oneOrThrowIfMany()?.let {
append("&genero=")
append(it.key)
}
filter.states.oneOrThrowIfMany()?.let {
append("&estado=")
append(
when (it) {
MangaState.ONGOING -> "ongoing"
MangaState.FINISHED -> "complete"
else -> ""
},
)
}
}
null -> {}
}
}
val doc = webClient.httpGet(url).parseHtml()
return doc.select("ul.animes li").map { li ->
val href = li.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(domain),
coverUrl = li.selectFirst("img")?.src().orEmpty(),
title = li.selectFirst(".title")?.text().orEmpty(),
altTitle = null,
rating = RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = null,
source = source,
isNsfw = isNsfwSource,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
val tags = webClient.httpGet("https://$domain/manga-list").parseHtml()
.selectLastOrThrow(".filter-bx .form-group select.custom-select").select("option").drop(1)
return tags.mapNotNullToSet { option ->
MangaTag(
key = option.attr("value").substringAfterLast("="),
title = option.text(),
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga {
val doc = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
val simpleDateFormat = SimpleDateFormat("dd/MM/yyyy", sourceLocale)
return manga.copy(
tags = doc.body().select(".genres a").mapNotNullToSet { a ->
MangaTag(
key = a.attr("href").substringAfterLast('='),
title = a.text(),
source = source,
)
},
description = doc.select(".sinopsis").html(),
state = when (doc.selectLast("span.anime-type-peli")?.text()?.lowercase()) {
"ongoing" -> MangaState.ONGOING
"complete" -> MangaState.FINISHED
else -> null
},
chapters = doc.select(".episodes-list li").mapChapters(reversed = true) { i, li ->
val url = li.selectFirstOrThrow("a").attrAsRelativeUrl("href")
MangaChapter(
id = generateUid(url),
name = li.selectFirstOrThrow(".flex-grow-1 span").text(),
number = i + 1,
url = url,
scanlator = null,
uploadDate = parseChapterDate(
simpleDateFormat,
li.selectLastOrThrow(".flex-grow-1 span").text(),
),
branch = null,
source = source,
)
},
)
}
private fun parseChapterDate(dateFormat: DateFormat, date: String?): Long {
val d = date?.lowercase() ?: return 0
return when {
d.endsWith(" ago") -> parseRelativeDate(date)
else -> dateFormat.tryParse(date)
}
}
private fun parseRelativeDate(date: String): Long {
val number = Regex("""(\d+)""").find(date)?.value?.toIntOrNull() ?: return 0
val cal = Calendar.getInstance()
return when {
WordSet("second").anyWordIn(date) -> cal.apply { add(Calendar.SECOND, -number) }.timeInMillis
WordSet("minute", "minutes").anyWordIn(date) -> cal.apply { add(Calendar.MINUTE, -number) }.timeInMillis
WordSet("hour", "hours").anyWordIn(date) -> cal.apply { add(Calendar.HOUR, -number) }.timeInMillis
WordSet("day", "days").anyWordIn(date) -> cal.apply { add(Calendar.DAY_OF_MONTH, -number) }.timeInMillis
WordSet("week", "weeks").anyWordIn(date) -> cal.apply { add(Calendar.WEEK_OF_YEAR, -number) }.timeInMillis
WordSet("month", "months").anyWordIn(date) -> cal.apply { add(Calendar.MONTH, -number) }.timeInMillis
WordSet("year").anyWordIn(date) -> cal.apply { add(Calendar.YEAR, -number) }.timeInMillis
else -> 0
}
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val doc = webClient.httpGet(chapter.url.toAbsoluteUrl(domain)).parseHtml().requireElementById("chapter_imgs")
return doc.select("img").map { img ->
val url = img.src()?.toRelativeUrl(domain) ?: img.parseFailed("Image src not found")
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
)
}
}
}

View File

@@ -0,0 +1,104 @@
package org.koitharu.kotatsu.parsers.site.en
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaParser
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("PO2SCANS", "Po2Scans", "en")
internal class Po2Scans(context: MangaLoaderContext) : MangaParser(context, MangaSource.PO2SCANS) {
override val availableSortOrders: Set<SortOrder> = EnumSet.of(SortOrder.ALPHABETICAL)
override val configKeyDomain = ConfigKey.Domain("po2scans.com")
override suspend fun getList(offset: Int, filter: MangaListFilter?): List<Manga> {
if (offset > 0) {
return emptyList()
}
val url = buildString {
append("https://")
append(domain)
append("/series")
when (filter) {
is MangaListFilter.Search -> {
append("?search=")
append(filter.query.urlEncoded())
}
is MangaListFilter.Advanced -> {}
null -> {}
}
}
val doc = webClient.httpGet(url).parseHtml()
return doc.select(".series-list").map { div ->
val href = "/" + div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
title = div.selectFirstOrThrow("h2").text(),
altTitle = null,
url = href,
publicUrl = href.toAbsoluteUrl(domain),
rating = RATING_UNKNOWN,
isNsfw = false,
coverUrl = div.selectFirstOrThrow("img").src()?.toAbsoluteUrl(domain).orEmpty(),
tags = emptySet(),
state = null,
author = null,
source = source,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> = emptySet()
override suspend fun getDetails(manga: Manga): Manga {
val doc = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
val dateFormat = SimpleDateFormat("dd MMM, yy", Locale.ENGLISH)
return manga.copy(
altTitle = null,
state = when (doc.select(".status span").last()?.text()) {
"Ongoing" -> MangaState.ONGOING
"Done" -> MangaState.FINISHED
else -> null
},
tags = emptySet(),
author = doc.select(".author span").last()?.text(),
description = doc.selectFirstOrThrow(".summary").text(),
chapters = doc.select(".chap-section .chap")
.mapChapters(reversed = true) { i, div ->
val a = div.selectFirstOrThrow("a")
val url = "/" + a.attrAsRelativeUrl("href").toAbsoluteUrl(domain)
MangaChapter(
id = generateUid(url),
name = a.text(),
number = i + 1,
url = url,
scanlator = null,
uploadDate = dateFormat.tryParse(div.select(".detail span").last()?.text()),
branch = null,
source = source,
)
},
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
return doc.select(".swiper-slide img").map { img ->
val url = img.src()?.replace("./assets", "/assets")?.toRelativeUrl(domain)
?: img.parseFailed("Image src not found")
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
)
}
}
}

View File

@@ -0,0 +1,176 @@
package org.koitharu.kotatsu.parsers.site.en
import androidx.collection.ArraySet
import kotlinx.coroutines.async
import kotlinx.coroutines.awaitAll
import kotlinx.coroutines.coroutineScope
import org.jsoup.nodes.Element
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.util.*
@MangaSourceParser("PURURIN", "Pururin", "en", ContentType.HENTAI)
internal class Pururin(context: MangaLoaderContext) :
PagedMangaParser(context, MangaSource.PURURIN, pageSize = 20) {
override val availableSortOrders: Set<SortOrder> =
EnumSet.of(SortOrder.UPDATED, SortOrder.POPULARITY, SortOrder.RATING, SortOrder.ALPHABETICAL)
override val configKeyDomain = ConfigKey.Domain("pururin.to")
override val isMultipleTagsSupported = false
override suspend fun getListPage(page: Int, filter: MangaListFilter?): List<Manga> {
val url = buildString {
append("https://")
append(domain)
when (filter) {
is MangaListFilter.Search -> {
append("/search?q=")
append(filter.query.urlEncoded())
append("&page=")
append(page.toString())
}
is MangaListFilter.Advanced -> {
append("/browse")
filter.tags.oneOrThrowIfMany()?.let {
append("/tags/content/")
append(it.key)
append("/")
}
append("?page=")
append(page)
append("&sort=")
when (filter.sortOrder) {
SortOrder.UPDATED -> append("")
SortOrder.POPULARITY -> append("most-viewed")
SortOrder.RATING -> append("highest-rated")
SortOrder.ALPHABETICAL -> append("title")
else -> append("")
}
}
null -> {
append("/browse?page=")
append(page)
}
}
}
val doc = webClient.httpGet(url).parseHtml()
return doc.select(".row-gallery a.card-gallery").map { a ->
val href = a.attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(domain),
coverUrl = a.selectFirst("img.card-img-top")?.src().orEmpty(),
title = a.selectFirst(".title")?.text().orEmpty(),
altTitle = null,
rating = RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = null,
source = source,
isNsfw = isNsfwSource,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
return coroutineScope {
(1..4).map { page ->
async { getTags(page) }
}
}.awaitAll().flattenTo(ArraySet(360))
}
private suspend fun getTags(page: Int): Set<MangaTag> {
val url = "https://$domain/tags/content?order=uses&page=$page"
val root = webClient.httpGet(url).parseHtml()
return root.parseTags()
}
private fun Element.parseTags() = select("table tr td a").mapToSet {
val href = it.attr("href").substringAfterLast("content/").substringBeforeLast('/')
MangaTag(
key = href,
title = it.text(),
source = source,
)
}
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val fullUrl = manga.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
manga.copy(
description = doc.selectFirst("p.mb-2")?.text().orEmpty(),
rating = doc.selectFirst("td span.rating")?.attr("content")?.toFloatOrNull()?.div(5f) ?: RATING_UNKNOWN,
tags = doc.body().select("tr:contains(Contents) ul.list-inline a").mapNotNullToSet {
val href = it.attr("href").substringAfterLast("content/").substringBeforeLast('/')
MangaTag(
key = href,
title = it.text(),
source = source,
)
},
author = doc.selectFirst("a[itemprop=author]")?.text(),
chapters = listOf(
MangaChapter(
id = manga.id,
name = manga.title,
number = 1,
url = manga.url,
scanlator = null,
uploadDate = 0,
branch = null,
source = source,
),
),
)
}
override suspend fun getRelatedManga(seed: Manga): List<Manga> {
val doc = webClient.httpGet(seed.url.toAbsoluteUrl(domain)).parseHtml()
val root = doc.body().selectFirstOrThrow(".row-gallery-small")
return root.select("a.card-gallery").mapNotNull { a ->
val href = a.attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(domain),
coverUrl = a.selectFirst("img.card-img-top")?.src().orEmpty(),
title = a.selectFirst(".title")?.text().orEmpty(),
altTitle = null,
rating = RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = null,
source = source,
isNsfw = false,
)
}
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
return doc.select(".gallery-preview img").map { url ->
val img = url.src()?.toRelativeUrl(domain) ?: url.parseFailed("Image src not found")
val urlImage = img.replace("t.", ".")
MangaPage(
id = generateUid(urlImage),
url = urlImage,
preview = null,
source = source,
)
}
}
}

View File

@@ -0,0 +1,150 @@
package org.koitharu.kotatsu.parsers.site.en
import okhttp3.Headers
import org.jsoup.nodes.Document
import org.koitharu.kotatsu.parsers.ErrorMessages
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.network.UserAgents
import org.koitharu.kotatsu.parsers.util.*
import java.text.DateFormat
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("REAPERCOMICS", "ReaperComics", "en")
internal class ReaperComics(context: MangaLoaderContext) :
PagedMangaParser(context, MangaSource.REAPERCOMICS, pageSize = 30) {
override val availableSortOrders: Set<SortOrder> = EnumSet.of(SortOrder.UPDATED, SortOrder.ALPHABETICAL)
override val configKeyDomain = ConfigKey.Domain("reaperscans.com")
override val isSearchSupported = false
override val headers: Headers = Headers.Builder()
.add("User-Agent", UserAgents.CHROME_DESKTOP)
.build()
override suspend fun getListPage(page: Int, filter: MangaListFilter?): List<Manga> {
val url = buildString {
append("https://")
append(domain)
when (filter) {
is MangaListFilter.Search -> {
throw IllegalArgumentException(ErrorMessages.SEARCH_NOT_SUPPORTED) // TODO
}
is MangaListFilter.Advanced -> {
append("/")
if (filter.sortOrder == SortOrder.UPDATED) {
append("latest/")
}
append("comics?page=")
append(page.toString())
}
null -> {
append("/latest/comics?page=")
append(page.toString())
}
}
}
return parseMangaList(webClient.httpGet(url).parseHtml())
}
private fun parseMangaList(docs: Document): List<Manga> {
return docs.select("main div.relative, main li.col-span-1").map {
val a = it.selectFirstOrThrow("a")
val url = a.attrAsAbsoluteUrl("href")
Manga(
id = generateUid(url),
url = url,
title = (it.selectFirst("p a") ?: it.selectLast("a"))?.text().orEmpty(),
altTitle = null,
publicUrl = url,
rating = RATING_UNKNOWN,
isNsfw = isNsfwSource,
coverUrl = it.selectFirstOrThrow("img").src().orEmpty(),
tags = emptySet(),
state = null,
author = null,
source = source,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> = emptySet()
override suspend fun getDetails(manga: Manga): Manga {
val doc = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
val simpleDateFormat = SimpleDateFormat("dd/MM/yyyy", sourceLocale)
return manga.copy(
description = doc.selectFirst("div.p-4 p.prose")?.html(),
state = when (doc.selectFirst("dl.mt-2 div:contains(Status) dd")?.text()?.lowercase()) {
"ongoing" -> MangaState.ONGOING
"complete" -> MangaState.FINISHED
else -> null
},
chapters = doc.select("div.p-2 div.pb-4 ul li").mapChapters(reversed = true) { i, li ->
val a = li.selectFirstOrThrow("a")
val chapterUrl = a.attrAsAbsoluteUrl("href").toRelativeUrl(domain)
MangaChapter(
id = generateUid(chapterUrl),
name = li.selectFirst("div.truncate p.truncate")?.text().orEmpty(),
number = i + 1,
url = chapterUrl,
scanlator = null,
uploadDate = parseChapterDate(
simpleDateFormat,
li.selectFirst("div.truncate div.items-center")?.text(),
),
branch = null,
source = source,
)
},
)
}
private fun parseChapterDate(dateFormat: DateFormat, date: String?): Long {
val d = date?.lowercase() ?: return 0
return when {
d.endsWith(" ago") -> parseRelativeDate(date)
else -> dateFormat.tryParse(date)
}
}
private fun parseRelativeDate(date: String): Long {
val number = Regex("""(\d+)""").find(date)?.value?.toIntOrNull() ?: return 0
val cal = Calendar.getInstance()
return when {
WordSet("second").anyWordIn(date) -> cal.apply { add(Calendar.SECOND, -number) }.timeInMillis
WordSet("minute", "minutes").anyWordIn(date) -> cal.apply { add(Calendar.MINUTE, -number) }.timeInMillis
WordSet("hour", "hours").anyWordIn(date) -> cal.apply { add(Calendar.HOUR, -number) }.timeInMillis
WordSet("day", "days").anyWordIn(date) -> cal.apply { add(Calendar.DAY_OF_MONTH, -number) }.timeInMillis
WordSet("week", "weeks").anyWordIn(date) -> cal.apply { add(Calendar.WEEK_OF_YEAR, -number) }.timeInMillis
WordSet("month", "months").anyWordIn(date) -> cal.apply { add(Calendar.MONTH, -number) }.timeInMillis
WordSet("year").anyWordIn(date) -> cal.apply { add(Calendar.YEAR, -number) }.timeInMillis
else -> 0
}
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val doc = webClient.httpGet(chapter.url.toAbsoluteUrl(domain)).parseHtml()
return doc.select("img.max-w-full").map { img ->
val url = img.src()?.toRelativeUrl(domain) ?: img.parseFailed("Image src not found")
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
)
}
}
}

View File

@@ -0,0 +1,142 @@
package org.koitharu.kotatsu.parsers.site.es
import kotlinx.coroutines.coroutineScope
import okhttp3.Headers
import org.jsoup.nodes.Document
import org.koitharu.kotatsu.parsers.ErrorMessages
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.network.UserAgents
import org.koitharu.kotatsu.parsers.util.*
import java.util.*
@MangaSourceParser("TEMPLESCANESP", "TempleScanEsp", "es", ContentType.HENTAI)
internal class TempleScanEsp(context: MangaLoaderContext) :
PagedMangaParser(context, MangaSource.TEMPLESCANESP, pageSize = 15) {
override val availableSortOrders: Set<SortOrder> = EnumSet.of(SortOrder.NEWEST, SortOrder.UPDATED)
override val configKeyDomain = ConfigKey.Domain("templescanesp.net")
override val isSearchSupported = false
override val headers: Headers = Headers.Builder()
.add("User-Agent", UserAgents.CHROME_DESKTOP)
.build()
override suspend fun getListPage(page: Int, filter: MangaListFilter?): List<Manga> {
val url = buildString {
append("https://")
append(domain)
when (filter) {
is MangaListFilter.Search -> {
throw IllegalArgumentException(ErrorMessages.SEARCH_NOT_SUPPORTED)
}
is MangaListFilter.Advanced -> {
if (filter.sortOrder == SortOrder.NEWEST) {
append("/comics?page=")
append(page.toString())
} else {
if (page > 1) {
return emptyList()
}
}
}
null -> {
append("/comics?page=")
append(page.toString())
}
}
}
val doc = webClient.httpGet(url).parseHtml()
return doc.select("div.grid figure").ifEmpty {
doc.requireElementById("projectsDiv").select("figure")
}.map { div ->
val href = div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(div.host ?: domain),
coverUrl = div.selectFirst("img")?.src().orEmpty(),
title = div.selectFirst("figcaption")?.text().orEmpty(),
altTitle = null,
rating = RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = null,
source = source,
isNsfw = isNsfwSource,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> = emptySet()
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val fullUrl = manga.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
val chaptersDeferred = getChapters(doc)
manga.copy(
description = doc.requireElementById("section-sinopsis").html(),
chapters = chaptersDeferred,
)
}
private fun getChapters(doc: Document): List<MangaChapter> {
return doc.body().select("div.grid-capitulos div.contenedor-capitulo-miniatura")
.mapChapters(reversed = true) { i, div ->
val href = div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
val date = parseUploadDate(div.selectFirstOrThrow("time").text())
MangaChapter(
id = generateUid(href),
name = div.requireElementById("name").text(),
number = i + 1,
url = href,
uploadDate = date,
source = source,
scanlator = null,
branch = null,
)
}
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
return doc.select("main.contenedor-imagen img").map { url ->
val img = url.src()?.toRelativeUrl(domain) ?: url.parseFailed("Image src not found")
MangaPage(
id = generateUid(img),
url = img,
preview = null,
source = source,
)
}
}
private fun parseUploadDate(timeStr: String?): Long {
timeStr ?: return 0
val timeWords = timeStr.split(' ')
if (timeWords.size != 3) return 0
val timeWord = timeWords[1]
val timeAmount = timeWords[0].toIntOrNull() ?: return 0
val timeUnit = when (timeWord) {
"minute", "minutes" -> Calendar.MINUTE
"hour", "hours" -> Calendar.HOUR
"day", "days" -> Calendar.DAY_OF_YEAR
"week", "weeks" -> Calendar.WEEK_OF_YEAR
"month", "months" -> Calendar.MONTH
"year", "years" -> Calendar.YEAR
else -> return 0
}
val cal = Calendar.getInstance()
cal.add(timeUnit, -timeAmount)
return cal.time.time
}
}

View File

@@ -0,0 +1,255 @@
package org.koitharu.kotatsu.parsers.site.es
import okhttp3.Headers
import okhttp3.HttpUrl.Companion.toHttpUrl
import org.jsoup.nodes.Document
import org.jsoup.nodes.Element
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("TUMANGAONLINE", "TuMangaOnline", "es")
class TuMangaOnlineParser(context: MangaLoaderContext) : PagedMangaParser(
context,
source = MangaSource.TUMANGAONLINE,
pageSize = 24,
) {
override val configKeyDomain = ConfigKey.Domain("visortmo.com")
private val chapterDateFormat = SimpleDateFormat("yyyy-MM-dd", sourceLocale)
override val availableContentRating: Set<ContentRating> = EnumSet.of(ContentRating.SAFE, ContentRating.ADULT)
override val availableSortOrders: Set<SortOrder> = EnumSet.of(
SortOrder.ALPHABETICAL,
SortOrder.ALPHABETICAL_DESC,
SortOrder.UPDATED,
SortOrder.NEWEST,
SortOrder.POPULARITY,
SortOrder.RATING,
)
override suspend fun getListPage(page: Int, filter: MangaListFilter?): List<Manga> {
val url = buildString {
append("https://")
append(domain)
append("/library")
when (filter) {
is MangaListFilter.Search -> {
append("?title=")
append(filter.query.urlEncoded())
}
is MangaListFilter.Advanced -> {
append("?order_item=")
append(
when (filter.sortOrder) {
SortOrder.POPULARITY -> "likes_count&order_dir=desc"
SortOrder.UPDATED -> "release_date&order_dir=desc"
SortOrder.NEWEST -> "creation&order_dir=desc"
SortOrder.ALPHABETICAL -> "alphabetically&order_dir=asc"
SortOrder.ALPHABETICAL_DESC -> "alphabetically&order_dir=desc"
SortOrder.RATING -> "score&order_dir=desc"
},
)
append("&filter_by=title")
if (filter.tags.isNotEmpty()) {
for (tag in filter.tags) {
append("&genders[]=")
append(tag.key)
}
}
filter.contentRating.oneOrThrowIfMany()?.let {
append("&erotic=")
append(
when (it) {
ContentRating.SAFE -> "false"
ContentRating.ADULT -> "true"
else -> ""
},
)
}
}
null -> {
append("?order_item=release_date&order_dir=desc&filter_by=title")
}
}
append("&_pg=1&page=")
append(page.toString())
}
val doc = webClient.httpGet(url, headers).parseHtml()
val items = doc.body().select("div.element")
return items.mapNotNull { item ->
val href =
item.selectFirst("a")?.attrAsRelativeUrlOrNull("href")?.substringAfter(' ') ?: return@mapNotNull null
Manga(
id = generateUid(href),
title = item.selectFirst("h4.text-truncate")?.text() ?: return@mapNotNull null,
coverUrl = item.select("style").toString().substringAfter("('").substringBeforeLast("')"),
altTitle = null,
author = null,
rating = item.selectFirst("span.score")?.text()?.toFloatOrNull()?.div(10F) ?: RATING_UNKNOWN,
url = href,
isNsfw = item.select("i").hasClass("fas fa-heartbeat fa-2x"),
tags = emptySet(),
state = null,
publicUrl = href.toAbsoluteUrl(doc.host ?: domain),
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga {
val doc = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
val contents = doc.body().selectFirstOrThrow("section.element-header-content")
return manga.copy(
description = contents.selectFirst("p.element-description")?.html(),
tags = contents.select("h6 a").mapNotNullToSet { a ->
MangaTag(
key = a.attr("href").substringBefore("&").substringAfterLast("="),
title = a.text(),
source = source,
)
},
largeCoverUrl = contents.selectFirst(".book-thumbnail")?.attrAsAbsoluteUrlOrNull("src"),
state = parseStatus(contents.select("span.book-status").text().orEmpty()),
author = contents.selectFirst("h5.card-title")?.attr("title")?.substringAfter(", "),
chapters = if (doc.select("div.chapters").isEmpty()) {
doc.select(oneShotChapterListSelector).mapChapters(reversed = true) { _, item ->
oneShotChapterFromElement(item)
}
} else {
val chapters = ChaptersListBuilder(10)
doc.select(regularChapterListSelector).reversed().forEachIndexed { i, item ->
val chapterName = item.select("div.col-10.text-truncate").text().replace("&nbsp;", " ").trim()
val scanElement = item.select("ul.chapter-list > li")
scanElement.forEach { chapters.add(regularChapterFromElement(it, chapterName, i)) }
}
chapters.toList()
},
)
}
private val oneShotChapterListSelector = "div.chapter-list-element > ul.list-group li.list-group-item"
private fun oneShotChapterFromElement(element: Element): MangaChapter {
val href = element.selectFirstOrThrow("div.row > .text-right > a").attrAsRelativeUrl("href")
return MangaChapter(
id = generateUid(href),
name = "One Shot",
number = 1,
url = href,
scanlator = element.select("div.col-md-6.text-truncate").text(),
branch = null,
uploadDate = chapterDateFormat.tryParse(element.select("span.badge.badge-primary.p-2").first()?.text()),
source = source,
)
}
private val regularChapterListSelector = "div.chapters > ul.list-group li.p-0.list-group-item"
private fun regularChapterFromElement(element: Element, chName: String, number: Int): MangaChapter {
val href = element.selectFirstOrThrow("div.row > .text-right > a").attrAsRelativeUrl("href")
return MangaChapter(
id = generateUid(href),
name = chName,
number = number + 1,
url = href,
scanlator = element.select("div.col-md-6.text-truncate").text(),
branch = null,
uploadDate = chapterDateFormat.tryParse(element.select("span.badge.badge-primary.p-2").first()?.text()),
source = source,
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val redirectDoc = webClient.httpGet(chapter.url.toAbsoluteUrl(domain), headers).parseHtml()
var doc = redirectToReadingPage(redirectDoc)
val currentUrl = doc.location()
val newUrl = if (!currentUrl.contains("cascade")) {
currentUrl.substringBefore("paginated") + "cascade"
} else {
currentUrl
}
if (currentUrl != newUrl) {
doc = webClient.httpGet(newUrl, headers).parseHtml()
}
return doc.select("div.viewer-container img:not(noscript img)").map {
val href = if (it.hasAttr("data-src")) {
it.attr("abs:data-src")
} else {
it.attr("abs:src")
}
MangaPage(
id = generateUid(href),
url = href,
preview = null,
source = source,
)
}
}
private suspend fun redirectToReadingPage(document: Document): Document {
val script1 = document.selectFirst("script:containsData(uniqid)")
val script2 = document.selectFirst("script:containsData(window.location.replace)")
val redirectHeaders = Headers.Builder().set("Referer", document.baseUri()).build()
if (script1 != null) {
val data = script1.data()
@Suppress("RegExpRedundantEscape")
val regexParams = """\{uniqid:\s*'(\S+)',\s*cascade:\s*(\S+)\}""".toRegex()
val regexAction = """form\.action\s?=\s?'(.+)'""".toRegex()
val params = regexParams.find(data)!!
val action = regexAction.find(data)!!.groupValues[1].toHttpUrl()
val formBody = mapOf(
"uniqid" to params.groupValues[1],
"cascade" to params.groupValues[2],
)
return redirectToReadingPage(webClient.httpPost(action, formBody, redirectHeaders).parseHtml())
}
if (script2 != null) {
val data = script2.data()
val regexRedirect = """window\.location\.replace\('(.+)'\)""".toRegex()
val url = regexRedirect.find(data)!!.groupValues[1]
return redirectToReadingPage(webClient.httpGet(url, redirectHeaders).parseHtml())
}
return document
}
override suspend fun getAvailableTags(): Set<MangaTag> {
val doc = webClient.httpGet("https://$domain/library", headers).parseHtml()
val elements = doc.body().select("div#books-genders > div > div")
return elements.mapNotNullToSet { element ->
MangaTag(
title = element.select("label").text(),
key = element.select("input").attr("value"),
source = source,
)
}
}
private fun parseStatus(status: String) = when {
status.contains("Publicándose") -> MangaState.ONGOING
status.contains("Finalizado") -> MangaState.FINISHED
else -> null
}
}

View File

@@ -0,0 +1,320 @@
package org.koitharu.kotatsu.parsers.site.fmreader
import kotlinx.coroutines.async
import kotlinx.coroutines.coroutineScope
import org.jsoup.nodes.Document
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.text.DateFormat
import java.text.SimpleDateFormat
import java.util.*
internal abstract class FmreaderParser(
context: MangaLoaderContext,
source: MangaSource,
domain: String,
pageSize: Int = 20,
) : PagedMangaParser(context, source, pageSize) {
override val configKeyDomain = ConfigKey.Domain(domain)
override val availableSortOrders: Set<SortOrder> = EnumSet.of(
SortOrder.UPDATED,
SortOrder.POPULARITY,
SortOrder.ALPHABETICAL,
SortOrder.ALPHABETICAL_DESC,
)
override val availableStates: Set<MangaState> = EnumSet.of(
MangaState.ONGOING,
MangaState.FINISHED,
MangaState.ABANDONED,
)
override val isTagsExclusionSupported = true
protected open val listUrl = "/manga-list.html"
protected open val datePattern = "MMMM d, yyyy"
protected open val tagPrefix = "manga-list-genre-"
init {
paginator.firstPage = 1
searchPaginator.firstPage = 1
}
@JvmField
protected val ongoing: Set<String> = setOf(
"on going",
"incomplete",
"en curso",
)
@JvmField
protected val finished: Set<String> = setOf(
"completed",
"completado",
)
@JvmField
protected val abandoned: Set<String> = hashSetOf(
"canceled",
"cancelled",
"drop",
)
override suspend fun getListPage(page: Int, filter: MangaListFilter?): List<Manga> {
val url = buildString {
append("https://")
append(domain)
append(listUrl)
append("?page=")
append(page.toString())
when (filter) {
is MangaListFilter.Search -> {
append("&name=")
append(filter.query.urlEncoded())
}
is MangaListFilter.Advanced -> {
append("&genre=")
append(filter.tags.joinToString(",") { it.key })
append("&ungenre=")
append(filter.tagsExclude.joinToString(",") { it.key })
append("&sort=")
when (filter.sortOrder) {
SortOrder.POPULARITY -> append("views")
SortOrder.UPDATED -> append("last_update")
SortOrder.ALPHABETICAL -> append("name&sort_type=ASC")
SortOrder.ALPHABETICAL_DESC -> append("name&sort_type=DESC")
else -> append("last_update")
}
append("&m_status=")
filter.states.oneOrThrowIfMany()?.let {
append(
when (it) {
MangaState.ONGOING -> "2"
MangaState.FINISHED -> "1"
MangaState.ABANDONED -> "3"
else -> ""
},
)
}
}
null -> append("&sort=last_update")
}
}
return parseMangaList(webClient.httpGet(url).parseHtml())
}
protected open fun parseMangaList(doc: Document): List<Manga> {
return doc.select("div.thumb-item-flow").map { div ->
val href = div.selectFirstOrThrow("div.series-title a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(div.host ?: domain),
coverUrl = div.selectFirstOrThrow("div.img-in-ratio").attr("data-bg")
?: div.selectFirstOrThrow("div.img-in-ratio").attr("style").substringAfter("(")
.substringBefore(")"),
title = div.selectFirstOrThrow("div.series-title").text().orEmpty(),
altTitle = null,
rating = RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = null,
source = source,
isNsfw = isNsfwSource,
)
}
}
protected open val selectBodyTag = "ul.filter-type li a"
override suspend fun getAvailableTags(): Set<MangaTag> {
val doc = webClient.httpGet("https://$domain/$listUrl").parseHtml()
return doc.select(selectBodyTag).mapNotNullToSet { a ->
val href = a.attr("href").substringAfter(tagPrefix).substringBeforeLast(".html")
MangaTag(
key = href,
title = a.text().toTitleCase(sourceLocale),
source = source,
)
}
}
protected open val selectDesc = "div.summary-content"
protected open val selectState = "ul.manga-info li:contains(Status) a"
protected open val selectAlt = "ul.manga-info li:contains(Other names)"
protected open val selectAut = "ul.manga-info li:contains(Author(s)) a"
protected open val selectTag = "ul.manga-info li:contains(Genre(s)) a"
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val fullUrl = manga.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
val chaptersDeferred = async { getChapters(doc) }
val desc = doc.selectFirst(selectDesc)?.html()
val stateDiv = doc.selectFirst(selectState)
val state = stateDiv?.let {
when (it.text().lowercase()) {
in ongoing -> MangaState.ONGOING
in finished -> MangaState.FINISHED
in abandoned -> MangaState.ABANDONED
else -> null
}
}
val alt = doc.body().selectFirst(selectAlt)?.text()?.replace("Other names", "")
val auth = doc.body().selectFirst(selectAut)?.text()
manga.copy(
tags = doc.body().select(selectTag).mapNotNullToSet { a ->
MangaTag(
key = a.attr("href").substringAfter(tagPrefix).substringBeforeLast(".html"),
title = a.text().toTitleCase(),
source = source,
)
},
description = desc,
altTitle = alt,
author = auth,
state = state,
chapters = chaptersDeferred.await(),
)
}
protected open val selectDate = "div.chapter-time"
protected open val selectChapter = "ul.list-chapters a"
protected open suspend fun getChapters(doc: Document): List<MangaChapter> {
val dateFormat = SimpleDateFormat(datePattern, sourceLocale)
return doc.body().select(selectChapter).mapChapters(reversed = true) { i, a ->
val href = a.attrAsRelativeUrl("href")
val dateText = a.selectFirst(selectDate)?.text()
MangaChapter(
id = generateUid(href),
name = a.selectFirstOrThrow("div.chapter-name").text(),
number = i + 1,
url = href,
uploadDate = parseChapterDate(
dateFormat,
dateText,
),
source = source,
scanlator = null,
branch = null,
)
}
}
protected open val selectPage = "div.chapter-content img"
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
return doc.select(selectPage).map { img ->
val url = img.src()?.toRelativeUrl(domain) ?: img.parseFailed("Image src not found")
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
)
}
}
protected fun parseChapterDate(dateFormat: DateFormat, date: String?): Long {
// Clean date (e.g. 5th December 2019 to 5 December 2019) before parsing it
val d = date?.lowercase() ?: return 0
return when {
d.endsWith(" ago") ||
d.endsWith(" atrás") ||
// short Hours
d.endsWith(" h") ||
// short Day
d.endsWith(" d") -> parseRelativeDate(date)
// Handle 'yesterday' and 'today', using midnight
d.startsWith("year") -> Calendar.getInstance().apply {
add(Calendar.DAY_OF_MONTH, -1) // yesterday
set(Calendar.HOUR_OF_DAY, 0)
set(Calendar.MINUTE, 0)
set(Calendar.SECOND, 0)
set(Calendar.MILLISECOND, 0)
}.timeInMillis
d.startsWith("today") -> Calendar.getInstance().apply {
set(Calendar.HOUR_OF_DAY, 0)
set(Calendar.MINUTE, 0)
set(Calendar.SECOND, 0)
set(Calendar.MILLISECOND, 0)
}.timeInMillis
date.contains(Regex("""\d(st|nd|rd|th)""")) -> date.split(" ").map {
if (it.contains(Regex("""\d\D\D"""))) {
it.replace(Regex("""\D"""), "")
} else {
it
}
}.let { dateFormat.tryParse(it.joinToString(" ")) }
else -> dateFormat.tryParse(date)
}
}
// Parses dates in this form:
// 21 hours ago
private fun parseRelativeDate(date: String): Long {
val number = Regex("""(\d+)""").find(date)?.value?.toIntOrNull() ?: return 0
val cal = Calendar.getInstance()
return when {
WordSet("second").anyWordIn(date) -> cal.apply { add(Calendar.SECOND, -number) }.timeInMillis
WordSet("min", "minute", "minutes", "minuto", "minutos").anyWordIn(date) -> cal.apply {
add(
Calendar.MINUTE,
-number,
)
}.timeInMillis
WordSet("hour", "hours", "hora", "horas", "h").anyWordIn(date) -> cal.apply {
add(
Calendar.HOUR,
-number,
)
}.timeInMillis
WordSet("day", "days", "día", "dia").anyWordIn(date) -> cal.apply {
add(
Calendar.DAY_OF_MONTH,
-number,
)
}.timeInMillis
WordSet("week", "weeks", "semana", "semanas").anyWordIn(date) -> cal.apply {
add(
Calendar.WEEK_OF_YEAR,
-number,
)
}.timeInMillis
WordSet("month", "months", "mes", "meses").anyWordIn(date) -> cal.apply {
add(
Calendar.MONTH,
-number,
)
}.timeInMillis
WordSet("year", "año", "años").anyWordIn(date) -> cal.apply { add(Calendar.YEAR, -number) }.timeInMillis
else -> 0
}
}
}

View File

@@ -0,0 +1,140 @@
package org.koitharu.kotatsu.parsers.site.fmreader.en
import kotlinx.coroutines.async
import kotlinx.coroutines.coroutineScope
import org.jsoup.nodes.Document
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.site.fmreader.FmreaderParser
import org.koitharu.kotatsu.parsers.util.*
import java.text.SimpleDateFormat
@MangaSourceParser("MANHWA18COM", "Manhwa18.com", "en", ContentType.HENTAI)
internal class Manhwa18Com(context: MangaLoaderContext) :
FmreaderParser(context, MangaSource.MANHWA18COM, "manhwa18.com") {
override val listUrl = "/tim-kiem"
override val selectState = "div.info-item:contains(Status) span.info-value "
override val selectAlt = "div.info-item:contains(Other name) span.info-value "
override val selectTag = "div.info-item:contains(Genre) span.info-value a"
override val datePattern = "dd/MM/yyyy"
override val selectPage = "div#chapter-content img"
override val selectBodyTag = "div.advanced-wrapper .genre_label"
override suspend fun getListPage(page: Int, filter: MangaListFilter?): List<Manga> {
val url = buildString {
append("https://")
append(domain)
append("/tim-kiem?page=")
append(page.toString())
when (filter) {
is MangaListFilter.Search -> {
append("&q=")
append(filter.query.urlEncoded())
}
is MangaListFilter.Advanced -> {
append("&accept_genres=")
append(filter.tags.joinToString(",") { it.key })
append("&reject_genres=")
append(filter.tagsExclude.joinToString(",") { it.key })
append("&sort=")
append(
when (filter.sortOrder) {
SortOrder.ALPHABETICAL -> "az"
SortOrder.ALPHABETICAL_DESC -> "za"
SortOrder.POPULARITY -> "top"
SortOrder.UPDATED -> "update"
SortOrder.NEWEST -> "new"
SortOrder.RATING -> "like"
},
)
filter.states.oneOrThrowIfMany()?.let {
append("&status=")
append(
when (it) {
MangaState.ONGOING -> "1"
MangaState.FINISHED -> "3"
MangaState.PAUSED -> "2"
else -> ""
},
)
}
}
null -> append("&sort=update")
}
}
return parseMangaList(webClient.httpGet(url).parseHtml())
}
override suspend fun getAvailableTags(): Set<MangaTag> {
val doc = webClient.httpGet("https://$domain/$listUrl").parseHtml()
return doc.select(selectBodyTag).mapNotNullToSet { label ->
val key = label.attr("data-genre-id")
MangaTag(
key = key,
title = label.selectFirstOrThrow(".gerne-name").text(),
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val fullUrl = manga.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
val chaptersDeferred = async { getChapters(doc) }
val desc = doc.selectFirstOrThrow(selectDesc).html()
val stateDiv = doc.selectFirst(selectState)
val state = stateDiv?.let {
when (it.text().lowercase()) {
in ongoing -> MangaState.ONGOING
in finished -> MangaState.FINISHED
else -> null
}
}
val alt = doc.body().selectFirst(selectAlt)?.text()?.replace("Other name", "")
val auth = doc.body().selectFirst(selectAut)?.text()
manga.copy(
tags = doc.body().select(selectTag).mapNotNullToSet { a ->
MangaTag(
key = a.attr("href").substringAfter("manga-list-genre-").substringBeforeLast(".html"),
title = a.text().toTitleCase(),
source = source,
)
},
description = desc,
altTitle = alt,
author = auth,
state = state,
chapters = chaptersDeferred.await(),
)
}
override suspend fun getChapters(doc: Document): List<MangaChapter> {
val dateFormat = SimpleDateFormat(datePattern, sourceLocale)
return doc.body().select(selectChapter).mapChapters(reversed = true) { i, a ->
val href = a.attrAsRelativeUrl("href")
val dateText = a.selectFirst(selectDate)?.text()?.substringAfter("- ")
MangaChapter(
id = generateUid(href),
name = a.selectFirstOrThrow("div.chapter-name").text(),
number = i + 1,
url = href,
uploadDate = parseChapterDate(
dateFormat,
dateText,
),
source = source,
scanlator = null,
branch = null,
)
}
}
}

View File

@@ -0,0 +1,114 @@
package org.koitharu.kotatsu.parsers.site.fmreader.es
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.site.fmreader.FmreaderParser
import org.koitharu.kotatsu.parsers.util.*
@MangaSourceParser("OLIMPOSCANS", "OlimpoScans", "es")
internal class OlimpoScans(context: MangaLoaderContext) :
FmreaderParser(context, MangaSource.OLIMPOSCANS, "olimposcans.com") {
override val selectState = "ul.manga-info li:contains(Estado) a"
override val selectAlt = "ul.manga-info li:contains(Otros nombres)"
override val selectTag = "ul.manga-info li:contains(Género) a"
override val tagPrefix = "lista-de-comics-genero-"
override val isMultipleTagsSupported = false
override val isTagsExclusionSupported = false
override suspend fun getListPage(page: Int, filter: MangaListFilter?): List<Manga> {
val url = buildString {
append("https://")
append(domain)
when (filter) {
is MangaListFilter.Search -> {
append(listUrl)
append("?page=")
append(page.toString())
append("&name=")
append(filter.query.urlEncoded())
}
is MangaListFilter.Advanced -> {
if (filter.tags.isNotEmpty()) {
filter.tags.oneOrThrowIfMany()?.let {
append("/lista-de-comics-genero-")
append(it.key)
append(".html")
}
} else {
append(listUrl)
append("?page=")
append(page.toString())
append("&sort=")
when (filter.sortOrder) {
SortOrder.POPULARITY -> append("views")
SortOrder.UPDATED -> append("last_update")
SortOrder.ALPHABETICAL -> append("name&sort_type=ASC")
SortOrder.ALPHABETICAL_DESC -> append("name&sort_type=DESC")
else -> append("last_update")
}
}
append("&m_status=")
filter.states.oneOrThrowIfMany()?.let {
append(
when (it) {
MangaState.ONGOING -> "2"
MangaState.FINISHED -> "1"
MangaState.ABANDONED -> "3"
else -> ""
},
)
}
}
null -> {
append(listUrl)
append("?page=")
append(page.toString())
append("&sort=last_update")
}
}
}
val doc = webClient.httpGet(url).parseHtml()
val lastPage =
doc.selectLast(".pagination a")?.attr("href")?.substringAfterLast("page=")?.substringBeforeLast("&artist")
?.toInt() ?: 1
if (lastPage < page) {
return emptyList()
}
return doc.select("div.thumb-item-flow").map { div ->
val href = "/" + div.selectFirstOrThrow("div.series-title a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(div.host ?: domain),
coverUrl = div.selectFirstOrThrow("div.img-in-ratio").attr("data-bg").toAbsoluteUrl(domain),
title = div.selectFirstOrThrow("div.series-title").text().orEmpty(),
altTitle = null,
rating = RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = null,
source = source,
isNsfw = isNsfwSource,
)
}
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = ("/" + chapter.url).toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
return doc.select(selectPage).map { img ->
val url = ("/proxy.php?link=" + img.src()).toRelativeUrl(domain)
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
)
}
}
}

View File

@@ -0,0 +1,85 @@
package org.koitharu.kotatsu.parsers.site.fmreader.ja
import org.jsoup.nodes.Document
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.site.fmreader.FmreaderParser
import org.koitharu.kotatsu.parsers.util.*
import java.text.SimpleDateFormat
@MangaSourceParser("KLZ9", "Klz9", "ja")
internal class Klz9(context: MangaLoaderContext) :
FmreaderParser(context, MangaSource.KLZ9, "klz9.com") {
override val selectDesc = "div.row:contains(Description)"
override val selectState = "ul.manga-info li:contains(Status) a"
override val selectAlt = "ul.manga-info li:contains(Other name (s))"
override val selectTag = "ul.manga-info li:contains(Genre(s)) a"
override val selectChapter = "tr"
override val selectDate = "td i"
override val selectPage = "img"
override val selectBodyTag = "div.panel-body a"
override fun parseMangaList(doc: Document): List<Manga> {
return doc.select("div.thumb-item-flow").map { div ->
val href = "/" + div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(div.host ?: domain),
coverUrl = div.selectFirstOrThrow("div.img-in-ratio").attr("style").substringAfter("('")
.substringBeforeLast("')"),
title = div.selectFirstOrThrow("div.series-title").text().orEmpty(),
altTitle = null,
rating = RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = null,
source = source,
isNsfw = isNsfwSource,
)
}
}
override suspend fun getChapters(doc: Document): List<MangaChapter> {
val slug = doc.selectFirstOrThrow("div.h0rating").attr("slug")
val docLoad =
webClient.httpGet("https://$domain/app/manga/controllers/cont.listChapter.php?slug=$slug").parseHtml()
val dateFormat = SimpleDateFormat(datePattern, sourceLocale)
return docLoad.body().select(selectChapter).mapChapters(reversed = true) { i, a ->
val href = "/" + a.selectFirstOrThrow("a.chapter").attrAsRelativeUrl("href")
val dateText = a.selectFirst(selectDate)?.text()
MangaChapter(
id = generateUid(href),
name = a.selectFirstOrThrow("a").text(),
number = i + 1,
url = href,
uploadDate = parseChapterDate(
dateFormat,
dateText,
),
source = source,
scanlator = null,
branch = null,
)
}
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
val cid = doc.selectFirstOrThrow("#chapter").attr("value")
val docLoad = webClient.httpGet("https://$domain/app/manga/controllers/cont.listImg.php?cid=$cid").parseHtml()
return docLoad.select(selectPage).map { img ->
val url = img.src()?.toRelativeUrl(domain) ?: img.parseFailed("Image src not found")
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
)
}
}
}

View File

@@ -0,0 +1,57 @@
package org.koitharu.kotatsu.parsers.site.fmreader.ja
import org.jsoup.nodes.Document
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.MangaChapter
import org.koitharu.kotatsu.parsers.model.MangaPage
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.site.fmreader.FmreaderParser
import org.koitharu.kotatsu.parsers.util.*
import java.text.SimpleDateFormat
@MangaSourceParser("WELOVEMANGA", "WeLoveManga", "ja")
internal class WeLoveManga(context: MangaLoaderContext) :
FmreaderParser(context, MangaSource.WELOVEMANGA, "welovemanga.one") {
override suspend fun getChapters(doc: Document): List<MangaChapter> {
val mid = doc.selectFirstOrThrow("div.cmt input").attr("value")
val docLoad =
webClient.httpGet("https://$domain/app/manga/controllers/cont.Listchapter.php?mid=$mid").parseHtml()
val dateFormat = SimpleDateFormat(datePattern, sourceLocale)
return docLoad.body().select(selectChapter).mapChapters(reversed = true) { i, a ->
val href = a.selectFirstOrThrow("a").attrAsRelativeUrl("href")
val dateText = a.selectFirst(selectDate)?.text()
MangaChapter(
id = generateUid(href),
name = a.selectFirstOrThrow("a").text(),
number = i + 1,
url = href,
uploadDate = parseChapterDate(
dateFormat,
dateText,
),
source = source,
scanlator = null,
branch = null,
)
}
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
val cid = doc.selectFirstOrThrow("#chapter").attr("value")
val docLoad = webClient.httpGet("https://$domain/app/manga/controllers/cont.listImg.php?cid=$cid").parseHtml()
return docLoad.select("img").map { img ->
val url = img.src()?.toRelativeUrl(domain) ?: img.parseFailed("Image src not found")
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
)
}
}
}

View File

@@ -0,0 +1,10 @@
package org.koitharu.kotatsu.parsers.site.fmreader.ja
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.site.fmreader.FmreaderParser
@MangaSourceParser("WELOMA", "Weloma", "ja")
internal class Weloma(context: MangaLoaderContext) :
FmreaderParser(context, MangaSource.WELOMA, "weloma.art")

View File

@@ -0,0 +1,191 @@
package org.koitharu.kotatsu.parsers.site.foolslide
import kotlinx.coroutines.coroutineScope
import org.json.JSONArray
import org.jsoup.nodes.Document
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.text.SimpleDateFormat
import java.util.*
internal abstract class FoolSlideParser(
context: MangaLoaderContext,
source: MangaSource,
domain: String,
pageSize: Int = 25,
) : PagedMangaParser(context, source, pageSize) {
override val configKeyDomain = ConfigKey.Domain(domain)
override val availableSortOrders: Set<SortOrder> = EnumSet.of(SortOrder.ALPHABETICAL)
protected open val listUrl = "directory/"
protected open val searchUrl = "search/"
protected open val pagination = true // false if the manga list has no pages
protected open val datePattern = "yyyy.MM.dd"
init {
paginator.firstPage = 1
searchPaginator.firstPage = 1
}
override suspend fun getListPage(page: Int, filter: MangaListFilter?): List<Manga> {
val doc =
when (filter) {
is MangaListFilter.Search -> {
if (page > 1) {
return emptyList()
}
val url = buildString {
append("https://")
append(domain)
append("/")
append(searchUrl)
}
webClient.httpPost(url, "search=${filter.query.urlEncoded()}").parseHtml()
}
is MangaListFilter.Advanced -> {
val url = buildString {
append("https://")
append(domain)
append("/")
append(listUrl)
// For some sites that don't have enough manga and page 2 links to page 1
if (!pagination) {
if (page > 1) {
return emptyList()
}
} else {
append(page.toString())
}
}
webClient.httpGet(url).parseHtml()
}
null -> {
val url = buildString {
append("https://")
append(domain)
append("/")
append(listUrl)
if (!pagination) {
if (page > 1) {
return emptyList()
}
} else {
append(page.toString())
}
}
webClient.httpGet(url).parseHtml()
}
}
return doc.select("div.list div.group").map { div ->
val href = div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(div.host ?: domain),
coverUrl = div.selectFirst("img")?.src().orEmpty(),// in search no img
title = div.selectFirstOrThrow(".title a").text().orEmpty(),
altTitle = null,
rating = RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = null,
source = source,
isNsfw = isNsfwSource,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> = emptySet()
protected open val selectInfo = "div.info"
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val fullUrl = manga.url.toAbsoluteUrl(domain)
val testAdultPage = webClient.httpGet(fullUrl).parseHtml()
val doc = if (testAdultPage.selectFirst("div.info form") != null) {
webClient.httpPost(fullUrl, "adult=true").parseHtml()
} else {
testAdultPage
}
val chapters = getChapters(doc)
val desc = if (doc.selectFirstOrThrow(selectInfo).html().contains("</b>")) {
doc.selectFirstOrThrow(selectInfo).text().substringAfterLast(": ")
} else {
doc.selectFirstOrThrow(selectInfo).text()
}
val author = if (doc.selectFirstOrThrow(selectInfo).html().contains("</b>")) {
doc.selectFirstOrThrow(selectInfo).text().substringAfter(": ").substringBefore("Art")
} else {
null
}
manga.copy(
coverUrl = doc.selectFirst(".thumbnail img")?.src() ?: manga.coverUrl,
description = desc,
altTitle = null,
author = author,
state = null,
chapters = chapters,
)
}
protected open val selectDate = ".meta_r"
protected open val selectChapter = "div.list div.element"
protected open suspend fun getChapters(doc: Document): List<MangaChapter> {
val dateFormat = SimpleDateFormat(datePattern, sourceLocale)
return doc.body().select(selectChapter).mapChapters(reversed = true) { i, div ->
val a = div.selectFirstOrThrow(".title a")
val href = a.attrAsRelativeUrl("href")
val dateText = div.selectFirstOrThrow(selectDate).text().substringAfter(", ")
MangaChapter(
id = generateUid(href),
name = a.text(),
number = i + 1,
url = href,
uploadDate = if (div.selectFirstOrThrow(selectDate).text().contains(", ")) {
dateFormat.tryParse(dateText)
} else {
0
},
source = source,
scanlator = null,
branch = null,
)
}
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
val script = doc.selectFirstOrThrow("script:containsData(var pages = )")
val images = JSONArray(script.data().substringAfterLast("var pages = ").substringBefore(';'))
val pages = ArrayList<MangaPage>(images.length())
for (i in 0 until images.length()) {
val pageTake = images.getJSONObject(i)
pages.add(
MangaPage(
id = generateUid(pageTake.getString("url")),
url = pageTake.getString("url"),
preview = null,
source = source,
),
)
}
return pages
}
}

View File

@@ -0,0 +1,10 @@
package org.koitharu.kotatsu.parsers.site.foolslide.en
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.site.foolslide.FoolSlideParser
@MangaSourceParser("DEATHTOLLSCANS", "DeathTollScans", "en")
internal class Deathtollscans(context: MangaLoaderContext) :
FoolSlideParser(context, MangaSource.DEATHTOLLSCANS, "reader.deathtollscans.net", 26)

View File

@@ -0,0 +1,12 @@
package org.koitharu.kotatsu.parsers.site.foolslide.en
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.site.foolslide.FoolSlideParser
@MangaSourceParser("MANGATELLERS", "Mangatellers", "en")
internal class Mangatellers(context: MangaLoaderContext) :
FoolSlideParser(context, MangaSource.MANGATELLERS, "reader.mangatellers.gr") {
override val pagination = false
}

View File

@@ -0,0 +1,10 @@
package org.koitharu.kotatsu.parsers.site.foolslide.en
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.site.foolslide.FoolSlideParser
@MangaSourceParser("READER_EVILFLOWERS", "Evil Flowers", "en")
internal class ReaderEvilflowers(context: MangaLoaderContext) :
FoolSlideParser(context, MangaSource.READER_EVILFLOWERS, "reader.evilflowers.com")

View File

@@ -0,0 +1,44 @@
package org.koitharu.kotatsu.parsers.site.foolslide.en
import kotlinx.coroutines.coroutineScope
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.site.foolslide.FoolSlideParser
import org.koitharu.kotatsu.parsers.util.*
@MangaSourceParser("SEINAGI", "Seinagi", "en")
internal class Seinagi(context: MangaLoaderContext) :
FoolSlideParser(context, MangaSource.SEINAGI, "reader.seinagi.org.es") {
override val pagination = false
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val fullUrl = manga.url.toAbsoluteUrl(domain)
val testAdultPage = webClient.httpGet(fullUrl).parseHtml()
val doc = if (testAdultPage.selectFirst("div.info form") != null) {
webClient.httpPost(fullUrl, "adult=true").parseHtml()
} else {
testAdultPage
}
val chapters = getChapters(doc)
val desc = if (doc.selectFirstOrThrow(selectInfo).html().contains("Description")) {
doc.selectFirstOrThrow(selectInfo).text().substringAfter("Description: ").substringBefore("Readings")
} else {
doc.selectFirstOrThrow(selectInfo).text()
}
val author = if (doc.selectFirstOrThrow(selectInfo).html().contains("Author")) {
doc.selectFirstOrThrow(selectInfo).text().substringAfter("Author: ").substringBefore("Art")
} else {
null
}
manga.copy(
coverUrl = doc.selectFirst(".thumbnail img")?.src() ?: manga.coverUrl,
description = desc,
altTitle = null,
author = author,
state = null,
chapters = chapters,
)
}
}

View File

@@ -0,0 +1,13 @@
package org.koitharu.kotatsu.parsers.site.foolslide.es
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.site.foolslide.FoolSlideParser
@MangaSourceParser("MENUDO_FANSUB", "Menudo Fansub", "es")
internal class MenudoFansub(context: MangaLoaderContext) :
FoolSlideParser(context, MangaSource.MENUDO_FANSUB, "www.menudo-fansub.com", 25) {
override val searchUrl = "slide/search/"
override val listUrl = "slide/directory/"
}

View File

@@ -0,0 +1,42 @@
package org.koitharu.kotatsu.parsers.site.foolslide.es
import kotlinx.coroutines.coroutineScope
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.site.foolslide.FoolSlideParser
import org.koitharu.kotatsu.parsers.util.*
@MangaSourceParser("PZYKOSIS666HFANSUB", "Pzykosis666h Fansub", "es", ContentType.HENTAI)
internal class Pzykosis666hFansub(context: MangaLoaderContext) :
FoolSlideParser(context, MangaSource.PZYKOSIS666HFANSUB, "lector.pzykosis666hfansub.com") {
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val fullUrl = manga.url.toAbsoluteUrl(domain)
val testAdultPage = webClient.httpGet(fullUrl).parseHtml()
val doc = if (testAdultPage.selectFirst("div.info form") != null) {
webClient.httpPost(fullUrl, "adult=true").parseHtml()
} else {
testAdultPage
}
val chapters = getChapters(doc)
val desc = if (doc.selectFirstOrThrow(selectInfo).html().contains("Descripción")) {
doc.selectFirstOrThrow(selectInfo).text().substringAfter("Descripción: ").substringBefore("Lecturas")
} else {
doc.selectFirstOrThrow(selectInfo).text()
}
val author = if (doc.selectFirstOrThrow(selectInfo).html().contains("Author")) {
doc.selectFirstOrThrow(selectInfo).text().substringAfter("Author: ").substringBefore("Art")
} else {
null
}
manga.copy(
coverUrl = doc.selectFirst(".thumbnail img")?.src() ?: manga.coverUrl,
description = desc,
altTitle = null,
author = author,
state = null,
chapters = chapters,
)
}
}

View File

@@ -0,0 +1,48 @@
package org.koitharu.kotatsu.parsers.site.foolslide.es
import kotlinx.coroutines.coroutineScope
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.ContentType
import org.koitharu.kotatsu.parsers.model.Manga
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.site.foolslide.FoolSlideParser
import org.koitharu.kotatsu.parsers.util.domain
import org.koitharu.kotatsu.parsers.util.parseHtml
import org.koitharu.kotatsu.parsers.util.selectFirstOrThrow
import org.koitharu.kotatsu.parsers.util.src
import org.koitharu.kotatsu.parsers.util.toAbsoluteUrl
@MangaSourceParser("SEINAGIADULTO", "Seinagi Adulto", "es", ContentType.HENTAI)
internal class SeinagiAdulto(context: MangaLoaderContext) :
FoolSlideParser(context, MangaSource.SEINAGIADULTO, "adulto.seinagi.org.es") {
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val fullUrl = manga.url.toAbsoluteUrl(domain)
val testAdultPage = webClient.httpGet(fullUrl).parseHtml()
val doc = if (testAdultPage.selectFirst("div.info form") != null) {
webClient.httpPost(fullUrl, "adult=true").parseHtml()
} else {
testAdultPage
}
val chapters = getChapters(doc)
val desc = if (doc.selectFirstOrThrow(selectInfo).html().contains("Descripción")) {
doc.selectFirstOrThrow(selectInfo).text().substringAfter("Descripción: ").substringBefore("Lecturas")
} else {
doc.selectFirstOrThrow(selectInfo).text()
}
val author = if (doc.selectFirstOrThrow(selectInfo).html().contains("Author")) {
doc.selectFirstOrThrow(selectInfo).text().substringAfter("Author: ").substringBefore("Art")
} else {
null
}
manga.copy(
coverUrl = doc.selectFirst(".thumbnail img")?.src().orEmpty(),// for manga result on search
description = desc,
altTitle = null,
author = author,
state = null,
chapters = chapters,
)
}
}

View File

@@ -0,0 +1,14 @@
package org.koitharu.kotatsu.parsers.site.foolslide.fr
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.site.foolslide.FoolSlideParser
@MangaSourceParser("HNISCANTRAD", "HniScantrad", "fr")
internal class HniScantrad(context: MangaLoaderContext) :
FoolSlideParser(context, MangaSource.HNISCANTRAD, "hni-scantrad.com") {
override val pagination = false
override val searchUrl = "lel/search/"
override val listUrl = "lel/directory/"
}

View File

@@ -0,0 +1,12 @@
package org.koitharu.kotatsu.parsers.site.foolslide.it
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.site.foolslide.FoolSlideParser
@MangaSourceParser("POWERMANGA", "PowerManga", "it")
internal class PowerManga(context: MangaLoaderContext) :
FoolSlideParser(context, MangaSource.POWERMANGA, "reader.powermanga.org") {
override val pagination = false
}

Some files were not shown because too many files have changed in this diff Show More