548 Commits

Author SHA1 Message Date
11f110b0bb Merge pull request #385 from KotatsuApp/add-hiatus
add hiatus
2023-11-18 16:09:02 +02:00
devi
6c2a526aeb rename 2023-11-18 11:27:38 +01:00
43a8caa687 Advanced filtering support 2023-11-18 10:22:35 +02:00
devi
f3fc848b6c Minor daily update 2023-11-17 17:54:57 +01:00
devi
98f5d115b3 Minor daily update 2023-11-16 17:55:13 +01:00
devi
885d98cebd Minor daily update 2023-11-15 18:37:47 +01:00
devi
6f73ff23e8 Minor update 2023-11-14 20:48:06 +01:00
2cd2d7dbc4 Merge pull request #368 from deividgabrielpeira/master
Update Daily
2023-11-14 07:31:22 +02:00
devi
d3afdad95b Minor update 2023-11-12 12:35:41 +01:00
Deivid Gabriel Pereira de Oliveira
48ac46eb5c chance url VexManga : Source url changed #370 2023-11-11 14:52:13 +00:00
Deivid Gabriel Pereira de Oliveira
2d6e6b359b they'll shut down leitorizakaya, they'll switch to blackoutcomics, it makes more sense to put it on madara, because they've made their own website. 2023-11-11 14:00:31 +00:00
Deivid Gabriel Pereira de Oliveira
7462743be8 se não funciona, so na proxima 2023-11-10 15:01:07 +00:00
Deivid Gabriel Pereira de Oliveira
db2acb752a fix 2023-11-10 14:55:37 +00:00
Deivid Gabriel Pereira de Oliveira
335229e61b fix error thanks @ztimms73 2023-11-10 14:48:25 +00:00
Deivid Gabriel Pereira de Oliveira
2acdd66f70 fix again 2023-11-10 14:42:42 +00:00
Deivid Gabriel Pereira de Oliveira
d3bec9006e Fix uppercase in parser 2023-11-10 14:40:23 +00:00
Deivid Gabriel Pereira de Oliveira
44f70c1388 Merge branch 'KotatsuApp:master' into master 2023-11-10 11:08:33 -03:00
02463e5833 [MangaReader] Fix NetShield bypass 2023-11-10 13:06:13 +02:00
Deivid Gabriel Pereira de Oliveira
778d14a1a2 change mangaSource 2023-11-09 22:27:19 +00:00
Deivid Gabriel Pereira de Oliveira
8b3814c485 , 2023-11-09 20:21:28 +00:00
Deivid Gabriel Pereira de Oliveira
6bcd9dd65c mudou de nome 2023-11-09 20:18:26 +00:00
154ae09c6e [MangaReader] Add NetShield bypass (testing) #198 2023-11-07 18:33:22 +02:00
devi
58c9d14d42 Minor daily update 2023-11-06 20:24:12 +01:00
3e69b88518 Update dependencies 2023-11-06 10:42:04 +02:00
devi
9d3671e117 Minor daily update 2023-11-05 18:07:37 +01:00
devi
0731d2487a Minor daily update 2023-11-05 18:07:12 +01:00
devi
5e4dd82b8d Minor daily update 2023-11-04 18:56:46 +01:00
devi
9d33d1a560 Minor daily update 2023-11-03 18:28:00 +01:00
devi
e788cff04a Minor daily update 2023-11-02 19:36:26 +01:00
face1d5b26 Merge pull request #355 from KotatsuApp/GalleryAdultsParser
Add GalleryAdultsParser
2023-11-02 08:52:39 +02:00
devi
18f8472baa Add GalleryAdultsParser
reworking some sources
add sources
2023-11-01 21:28:36 +01:00
devi
f14edd871f Minor daily update 2023-11-01 17:22:52 +01:00
c892a38c52 Trigger rebuild 2023-11-01 16:25:49 +02:00
devi
e9d473711d Minor daily update
- Fix tag ninemanga
- Fix search neonep parsser
2023-10-31 19:03:28 +01:00
devi
97a8bb1ca7 Minor daily update 2023-10-30 18:57:42 +01:00
devi
66d47bf579 Minor daily update 2023-10-29 18:31:35 +01:00
devi
508647eb5a Minor daily update ( and remove warning for name ) 2023-10-28 22:22:01 +02:00
4ca3a492b0 [ComicK] Fix genres 2023-10-28 12:40:00 +03:00
devi
fffe104cbe Minor daily update 2023-10-27 18:48:33 +02:00
devi
6258476a58 Minor daily update 2023-10-26 18:11:42 +02:00
devi
af1aca8725 Merge remote-tracking branch 'origin/master' 2023-10-25 17:51:04 +02:00
devi
bdb7fbc039 Minor daily update 2023-10-25 17:50:58 +02:00
6bf0ae92e4 [*chan] Fix sort orders 2023-10-25 13:36:42 +03:00
17b8fd51b7 [ComicK] Fix chapters numbers 2023-10-25 12:53:48 +03:00
d089fad6ce [MangaLib] Add branching support 2023-10-25 12:48:44 +03:00
ec11b79b18 [MangaLib] Fix search result covers 2023-10-25 12:22:10 +03:00
0f216c6de9 [Remanga] Fix loading large amount of chapters 2023-10-25 12:09:41 +03:00
devi
0606ace83a Merge pull request #328 from deividgabrielpeira/master
HentaiGekkout(pt)
2023-10-24 17:04:56 +02:00
Deivid Gabriel Pereira de Oliveira
746d857550 I forgot about Hentai Gekkou, which is part of Gekkou, which was taken down together by the DMCA. 2023-10-24 12:04:43 +00:00
devi
cf1d0b8d7e Merge pull request #327 from deividgabrielpeira/master
GekkouScan(pt) and FinalScans(pt), is down
2023-10-24 08:20:42 +02:00
Deivid Gabriel Pereira de Oliveira
d94145c6e5 GekkouScan(pt), has been taken down by the DMCA
FinalScans(pt), just closed
2023-10-24 00:42:56 +00:00
devi
b71a0ac3cf Minor daily update 2023-10-23 18:24:51 +02:00
devi
32afa26be5 Minor daily update 2023-10-22 20:15:17 +02:00
devi
23cc86e429 Minor daily update 2023-10-21 21:20:48 +02:00
devi
87032a6835 Minor daily update 2023-10-20 15:28:14 +02:00
devi
a4d9c6e36c Minor daily update 2023-10-19 18:46:07 +02:00
devi
76af0b0edb Minor daily update 2023-10-17 18:49:33 +02:00
0054d06e6e Fix sources titles 2023-10-17 09:49:38 +03:00
devi
e2e027df81 Minor daily update 2023-10-16 19:54:49 +02:00
a61e441e79 Merge pull request #283 from KotatsuApp/devi4
add sources and fix
2023-10-16 12:39:36 +03:00
devi
9b4153aacc fix 2023-10-15 10:12:20 +02:00
devi
b1d2d0a475 add Fakku 2023-10-14 16:43:42 +02:00
devi
aeec98db2a Rework tag hentaivn 2023-10-14 10:58:47 +02:00
devi
00f21de9eb quick fix 2023-10-14 10:28:30 +02:00
devi
2d595bb6ee Add small sources, fix , rename sources , remove code not used, remove dead sources, typo. 2023-10-14 10:10:21 +02:00
devi
e43c9f6489 Add small sources, fix , rename sources , remove code not used, remove dead sources, typo. 2023-10-14 10:08:00 +02:00
devi
81004b53a9 add Yaoi Flix 2023-10-13 18:38:36 +02:00
devi
85a0918da5 add Manga Ay 2023-10-13 17:38:02 +02:00
devi
b10fa0fd10 add change requested and fix url 2023-10-13 16:15:19 +02:00
a2f4bb35cd [*chan] Fix title parsing 2023-10-13 13:34:07 +03:00
devi
4120514256 add search on manga online 2023-10-12 19:39:15 +02:00
devi
29ef717ece add sources and fix 2023-10-12 18:56:49 +02:00
devi
d34cfbf405 add sources and fix 2023-10-12 18:56:20 +02:00
devi
94c2326153 Rework Yugen Mangas 2023-10-11 19:49:59 +02:00
devi
06c2287b71 fix 2023-10-11 17:47:14 +02:00
devi
8cb0d3d6fe Tag reduction on Pururin 2023-10-10 17:24:22 +02:00
devi
76860f82b9 add sources and fix 2023-10-10 17:17:06 +02:00
devi
f134137a05 add sources and fix 2023-10-09 17:58:43 +02:00
devi
fc231ec330 fix Asura Scans 2023-10-08 21:28:13 +02:00
devi
d3c32faf01 Tag reduction on hentaifox 2023-10-08 18:38:32 +02:00
devi
d16f4cd892 add ImHentai 2023-10-08 17:31:01 +02:00
devi
9a8de28a1d add Pururin 2023-10-08 15:29:25 +02:00
devi
939f08b8a1 add Mirai Scans 2023-10-08 09:55:59 +02:00
devi
09500c0734 add sources and fix 2023-10-07 17:10:29 +02:00
400a90464e [LineWebtoons] Refactoring 2023-10-06 10:36:51 +03:00
9103f9d473 Merge pull request #273 from DCNick3/line-webtoons-source
Add Line Webtoons sources (webtoons.com)
2023-10-06 09:47:45 +03:00
devi
27c29f99d9 minor fixes and add small sources 2023-10-04 17:14:12 +02:00
Nikita Strygin
753c27a90c Clean up code after review
- Do not use ConfigKey.Domain for supplementary domains
- use IllegalArgumentException
- use "$var" instead of "${var}" where possible
- fix the language in `publicUrl`
- use `?` to handle `largeCoverUrl`
- use `requireNotNull` instead of `!!`
2023-10-04 15:19:27 +03:00
3d7e62d2fe [MintManga] Update domain 2023-10-04 14:02:09 +03:00
e8289277a7 Merge pull request #274 from KotatsuApp/devi3
Add sources
2023-10-04 13:21:54 +03:00
Nikita Strygin
887fc691d1 Add support for more languages in Line Webtoons parser
Also change the default sorting to POPULARITY & clean up
2023-10-04 12:03:21 +03:00
devi
543e3da194 Add sources 2023-10-03 18:25:43 +02:00
devi
908cc22619 Minor fix 2023-10-03 16:55:32 +02:00
Nikita Strygin
c09d484a14 Add Line Webtoons sources (webtoons.com)
It uses the API of mobile app
2023-10-03 09:15:39 +03:00
07c3f22f75 Merge pull request #268 from VietAnh14/source/yurineko 2023-10-02 20:08:08 +03:00
devi
d494bf19ee Small fix 2023-10-02 17:29:07 +02:00
vianh
eda4f75219 [Yurineko] Fix api domain and chapters order 2023-10-02 21:21:58 +07:00
931f126119 Merge pull request #269 from deividgabrielpeira/patch-1
prismascan chance Domain
2023-10-02 13:19:59 +03:00
bc171ce06e [Grouple] Fix pumpkins 2023-10-02 13:19:36 +03:00
Deivid Gabriel Pereira de Oliveira
191d1fcec8 prismascan chance Domain 2023-10-01 21:01:05 -03:00
vianh
eaea9e54d1 [Yurineko] New source 2023-10-01 20:27:51 +07:00
devi
c9b3912fd4 Quick small fix 2023-10-01 00:01:20 +02:00
1611dc3429 Remove duplicated classes 2023-09-30 17:24:47 +03:00
88dc2145e0 Merge branch 'AwkwardPeak7-ksk' 2023-09-30 17:09:27 +03:00
8979987344 Merge pull request #266 from KotatsuApp/devi2
Update focus on fix and remove dead sources
2023-09-30 17:05:04 +03:00
AwkwardPeak7
1cec2cc187 remove need for refreshing chapter list 2023-09-29 20:29:14 +05:00
devi
a1279b64a2 small fix and sources 2023-09-29 17:21:38 +02:00
AwkwardPeak7
2ce5f2563a KskMoe: get original quality images 2023-09-29 19:29:57 +05:00
devi
75a60ff738 fix Union Mangás 2023-09-28 20:04:46 +02:00
devi
74eb7f83d6 other fix 2023-09-28 17:24:02 +02:00
devi
5b2ac5f6fb typo 2023-09-27 18:51:10 +02:00
devi
fde4ba1d85 remove mangatone 2023-09-27 18:45:56 +02:00
devi
8278a8f5c3 multiples fix, remove dead source and add source 2023-09-27 18:44:09 +02:00
a39d644094 [Remanga] Ability to change user-agent 2023-09-27 17:11:00 +03:00
0004be15ba Merge branch 'source/devi' 2023-09-20 12:11:03 +03:00
8fef459346 Misc fixes 2023-09-20 12:09:54 +03:00
109603f124 [HentaiUkr] Fix ids 2023-09-20 10:04:15 +03:00
91879a8405 Merge pull request #261 from CakesTwix/master
[HentaiUkr] Tag ID is String now
2023-09-20 09:43:33 +03:00
devi
f471a3513e add mangashiro.me and fix 2023-09-19 18:48:26 +02:00
CakesTwix
7362111318 [HentaiUkr] Tag ID is String now 2023-09-19 11:29:52 +03:00
devi
609c7188c0 add sources tr 2023-09-18 19:48:24 +02:00
devi
a106b34379 add sources and fix 2023-09-18 18:39:05 +02:00
devi
853b95363e add missing content type 2023-09-17 19:49:21 +02:00
devi
45113e419e small fix 2023-09-17 17:18:09 +02:00
devi
519bb3053b add ksk 2023-09-17 16:57:50 +02:00
devi
366cc862ba remove 1stkissmanga.me and add likemanga 2023-09-17 13:12:35 +02:00
devi
611aaf0d38 remove mirrordesu 2023-09-17 10:52:27 +02:00
devi
712d829b54 add sources 2023-09-17 10:50:01 +02:00
devi
e25656c090 reformat 2023-09-17 09:35:58 +02:00
devi
7debe6d7e2 add sources 2023-09-16 17:07:59 +02:00
devi
55e14e4cb3 add sources 2023-09-16 17:07:22 +02:00
7fbeb2e266 Add gradle jvmargs 2023-09-12 19:22:36 +03:00
7be6d779da Reformat 2023-09-12 19:15:57 +03:00
96585fbdf7 [DesuMe] Add alternate domain 2023-09-12 18:27:04 +03:00
1a0d623c02 Merge pull request #252 from davvarrr/master
adds sources and fix
2023-09-12 18:25:55 +03:00
67cf2c2a6e Revert MangaSources changes 2023-09-12 18:24:13 +03:00
b6dd7fb5f6 [HentaiUkr] Fixes 2023-09-12 18:23:34 +03:00
e0c5562600 Merge pull request #251 from CakesTwix/master
New 18+ UA Source and update urls in `CONTRIBUTING.md`
2023-09-12 13:47:20 +03:00
devi
2447f66834 change domain 2023-09-10 09:33:10 +02:00
devi
11ed27e397 correction 2023-09-09 18:12:19 +02:00
devi
6158547571 correction 2023-09-09 18:10:16 +02:00
devi
a24dfde1ba correction 2023-09-09 18:09:58 +02:00
devi
d805b49a4d adds sources and fix 2023-09-09 17:12:20 +02:00
devi
f871bf77fa Merge remote-tracking branch 'origin/master' 2023-09-09 17:11:36 +02:00
devi
be610b825d adds sources and fix 2023-09-09 17:11:14 +02:00
f791cc6f9c Added SoftSuspendLazy class 2023-09-09 16:36:13 +03:00
CakesTwix
30b2e81b15 [CONTRIBUTING] Update example urls
380861f29e
2023-09-08 20:51:36 +03:00
CakesTwix
067c7f2d0d [HentaiUkr] New 18+ UA Source 2023-09-08 20:51:25 +03:00
aae3fa3b05 Add abandoned state 2023-09-08 13:19:26 +03:00
41cd8ac7d7 Merge pull request #250 from CakesTwix/master
[HoneyManga] Change search url
2023-09-08 13:09:34 +03:00
CakesTwix
61f2a499da [HoneyManga] Change search url 2023-09-08 12:46:18 +03:00
2f7e704e21 [MangaLib] Check page response size 2023-09-03 18:55:55 +03:00
4d22e7d1e8 Merge pull request #243 from davvarrr/master
Add sources and fix
2023-09-03 18:30:44 +03:00
devi
18125ea982 fix url Ozulscans 2023-09-03 09:52:55 +02:00
devi
1498541b45 fix url Nivera Fansub 2023-09-02 16:27:24 +02:00
devi
4faaa9fc13 fix url Manhwalist 2023-09-02 13:35:27 +02:00
devi
c6c4b7190a fix url Random Scans 2023-09-02 10:09:33 +02:00
devi
c0bf3ad56c add RackusReads 2023-09-01 12:28:10 +02:00
devi
ad51271c40 format code 2023-09-01 12:14:32 +02:00
devi
fce952d240 change request and change url Nettruyenmax & NetTruyen 2023-09-01 12:13:57 +02:00
devi
0dc5032a7a change url Shinigami 2023-08-31 19:22:51 +02:00
devi
0395f29503 small change 2023-08-31 12:31:28 +02:00
devi
03581add21 add Perf Scan and fix Legacy Scans 2023-08-31 12:26:13 +02:00
devi
e0249accf8 fix domain Mgkomik 2023-08-31 09:46:19 +02:00
devi
a3afbafdcc remove small code 2023-08-30 18:03:45 +02:00
devi
4d4c126327 remove dead source 2023-08-30 17:50:26 +02:00
devi
c4bf7b2ae0 add sources and fix 2023-08-30 13:45:40 +02:00
devi
5d5f804c76 add sources and fix 2023-08-30 13:44:25 +02:00
3a76504380 [NepNep] Fixes 2023-08-24 14:16:47 +03:00
5035c90c73 Merge pull request #238 from davvarrr/master
add source and move Manga4Life to tempalte parser
2023-08-24 13:55:16 +03:00
devi
7f1cd30ce8 change domaine for QueenScans 2023-08-21 20:49:36 +02:00
devi
155a91a13d add Mangarbic 2023-08-21 12:19:54 +02:00
devi
8c291e274d nothing 2023-08-21 10:36:38 +02:00
devi
10e1eb8aac add source and move Manga4Life to tempalte parser
and fix url Duniakomik
2023-08-21 10:33:47 +02:00
06a2aa6f97 [Manga4Life] Refactoring and fixes 2023-08-19 17:38:57 +03:00
f08a2e4551 Merge pull request #235 from davvarrr/master
add sources and fix
2023-08-19 17:18:05 +03:00
e6839b1270 [Grouple] Change getPageUrl exception 2023-08-19 16:53:42 +03:00
270890f841 [NudeMoon] Migrate to mobile website parsing 2023-08-19 16:51:03 +03:00
devi
02581c9e2d Change and add Manga4Life 2023-08-19 15:38:40 +02:00
devi
ea4e69df19 remove japscan 2023-08-19 08:40:03 +02:00
devi
cb0ebb31f7 add more source 2023-08-18 20:03:31 +02:00
devi
497cd0bd15 add more source 2023-08-18 19:58:01 +02:00
devi
b74c7841dd add sources and fix 2023-08-18 17:39:35 +02:00
407ef5b655 Merge pull request #232 from davvarrr/master
quick fix
2023-08-14 09:54:20 +03:00
devi
f034b1874b add Mangakakalot 2023-08-13 11:26:29 +02:00
devi
eac086b935 add jaiminisbox 2023-08-13 09:26:20 +02:00
devi
ea310b184e add MangaCrab and MantrazScan 2023-08-13 09:20:15 +02:00
devi
ea3386eda5 add MangaCrab and MantrazScan 2023-08-13 09:19:52 +02:00
devi
2efdb1940a fix Asura Scans 2023-08-13 09:00:57 +02:00
devi
896139a8ed Add exception for source don't support multi tags
and other fix
Add birdToon
2023-08-12 15:24:27 +02:00
devi
da344233f4 quick fix 2023-08-11 18:30:48 +02:00
9cf3551356 [HentaiVN] Fix async/await 2023-08-11 12:01:32 +03:00
10aaae1ca8 Merge pull request #229 from davvarrr/master
Add template FoolSlideParser ands sources
2023-08-11 11:44:19 +03:00
dcb5206272 Add minus operator to favicons 2023-08-11 11:20:27 +03:00
devi
ccdd2b6c19 fix TuMangaOnlineParser 2023-08-10 17:47:01 +02:00
devi
16f3e1c180 add Ninja Scan 2023-08-09 21:10:24 +02:00
devi
7283cde5ff add missing contenttype 2023-08-09 21:01:43 +02:00
devi
4c20384d39 add sources and small fix 2023-08-09 20:59:03 +02:00
devi
5e9bc4fcc7 fix async and remove .imageUrl() to add .src() 2023-08-09 19:49:19 +02:00
devi
e5448649c3 fix pagination HentaiVNParser 2023-08-07 22:27:18 +02:00
devi
951e89d429 add rating on MadaraParser 2023-08-07 19:24:56 +02:00
devi
f34dd7926a add skanlacje-feniksy and fix hentaivn 2023-08-07 17:44:25 +02:00
devi
b900c05d9b Merge remote-tracking branch 'origin/master' 2023-08-06 22:45:34 +02:00
devi
06948934ac and sources 2023-08-06 22:45:14 +02:00
devi
e82915ef17 Merge branch 'KotatsuApp:master' into master 2023-08-06 17:56:52 +02:00
devi
bb37da403f Add template FoolSlideParser ands sources 2023-08-06 17:56:17 +02:00
065263a9bd Merge pull request #227 from davvarrr/master 2023-08-06 10:35:45 +03:00
devi
f3359cc830 quick fix hentai20 move to tempalte mangareader 2023-08-06 09:22:00 +02:00
1b3b5d95c0 Merge pull request #226 from davvarrr/master
fix and add sources
2023-08-06 09:00:10 +03:00
c290ba5436 Update src/main/kotlin/org/koitharu/kotatsu/parsers/site/en/DynastyScans.kt 2023-08-06 09:00:04 +03:00
devi
bb4ea05375 fix domaine peacescans 2023-08-05 18:18:25 +02:00
devi
47dde65f65 fix and add sources 2023-08-05 18:04:13 +02:00
08eb9552d6 Update gradle, kotlin and enum extensions 2023-08-04 15:58:57 +03:00
6058221fec Merge pull request #223 from davvarrr/master
add templates and sources and fix
2023-08-02 10:05:16 +03:00
devi
5bf4ecb394 min change 2023-07-31 20:28:01 +02:00
devi
41fa80e8b8 Change .src() in accord of commit 3c43590 2023-07-31 20:22:56 +02:00
devi
0dd3a26080 no pagination 2023-07-31 20:12:37 +02:00
devi
fb400d83e3 q fix 2023-07-31 19:54:21 +02:00
devi
37e5c43e5c Merge branch 'KotatsuApp:master' into master 2023-07-31 19:50:21 +02:00
341aa0e13d [JapScan] Migrate to desktop version 2023-07-31 15:08:46 +03:00
3c4359096d Add Jsoup extensions 2023-07-31 13:01:59 +03:00
devi
d56240086c Update src/main/kotlin/org/koitharu/kotatsu/parsers/site/zmanga/ZMangaParser.kt
Co-authored-by: Koitharu <nvasya95@gmail.com>
2023-07-31 11:58:33 +02:00
devi
bfea5b314e Update src/main/kotlin/org/koitharu/kotatsu/parsers/site/fr/ScansMangasMe.kt
Co-authored-by: Koitharu <nvasya95@gmail.com>
2023-07-31 11:58:22 +02:00
03b4fc9f00 Add Result utils 2023-07-31 09:53:21 +03:00
devi
826563694e add MangaboxParser and sourcres 2023-07-30 21:53:10 +02:00
devi
880ff923bc add AnimeBootstrapParser and sources 2023-07-30 15:11:44 +02:00
devi
cd0d4b103a add zmanga and soruces 2023-07-30 12:31:28 +02:00
devi
fefec4985d add Wpcomics and sources 2023-07-29 16:18:58 +02:00
devi
ba655f3d64 add domain and fix search url 2023-07-29 09:30:44 +02:00
devi
be3812fa2f fix pagination komikcast 2023-07-29 09:04:25 +02:00
devi
87d4326411 add MadthemeParser and sources 2023-07-29 08:11:51 +02:00
devi
ef6ede6b12 add correct page for UPDATED 2023-07-28 17:07:29 +02:00
devi
eeccda24b0 Add 2 source more typo and add val on gettags on madara for site not let list /manga/ 2023-07-28 16:53:17 +02:00
devi
2bf29da2ea small fix and rename bad typo 2023-07-28 16:16:13 +02:00
devi
0b54029e26 Merge remote-tracking branch 'origin/master' 2023-07-28 15:58:14 +02:00
d24ab347cb Move CloudFlareProtectedException to test sources 2023-07-28 14:35:13 +03:00
74ffe9418b Fix html descriptions 2023-07-28 10:49:20 +03:00
c945434d71 Merge pull request #222 from davvarrr/master
add new template and source
2023-07-28 10:38:22 +03:00
devi
b28545c852 add sources 2023-07-27 20:50:53 +02:00
devi
c8d33d06a1 add fmreader 2023-07-27 20:27:56 +02:00
devi
2089c3cc7b fix import name 2023-07-27 17:16:27 +02:00
devi
5df1da2bfd fix nswf 2023-07-27 16:52:56 +02:00
devi
70fda48983 fix nswf 2023-07-27 16:52:31 +02:00
devi
36fe5cb625 Merge branch 'KotatsuApp:master' into master 2023-07-27 16:45:45 +02:00
devi
7e2a00520b fix and add updated on template Mmrcms 2023-07-27 16:42:28 +02:00
caeeb699d1 Add content type field to sources 2023-07-27 16:33:46 +03:00
devi
f6dc4f9a00 add source 2023-07-26 19:04:43 +02:00
devi
28135aed66 add 2 new template and new source 2023-07-25 21:30:07 +02:00
8e452f4271 [MangaLib] Fix Android compatibility 2023-07-24 20:09:27 +03:00
f4656efebd Update packages structure 2023-07-24 20:02:08 +03:00
42cc0430f8 Merge pull request #220 from davvarrr/master
fix some source and add source
2023-07-24 15:54:21 +03:00
81dd8b58c8 Merge pull request #219 from VietAnh14/source/hentaivn
Add new source HentaiVN
2023-07-24 15:53:38 +03:00
415da81b1a [Komikcast] Fix chatpers names #221 2023-07-24 11:07:38 +03:00
74a20455f2 [MangaLib] Fix pages url 2023-07-24 11:06:36 +03:00
devi
0bdce39def Update src/main/kotlin/org/koitharu/kotatsu/parsers/site/vi/LxManga.kt
Co-authored-by: Koitharu <nvasya95@gmail.com>
2023-07-24 08:50:40 +02:00
devi
24f7d8b5bd Update src/main/kotlin/org/koitharu/kotatsu/parsers/site/fr/LugnicaScans.kt
Co-authored-by: Koitharu <nvasya95@gmail.com>
2023-07-24 08:50:33 +02:00
devi
18adc66f53 Update src/main/kotlin/org/koitharu/kotatsu/parsers/site/fr/LugnicaScans.kt
Co-authored-by: Koitharu <nvasya95@gmail.com>
2023-07-24 08:50:26 +02:00
devi
8c1c855531 Update src/main/kotlin/org/koitharu/kotatsu/parsers/site/fr/LugnicaScans.kt
Co-authored-by: Koitharu <nvasya95@gmail.com>
2023-07-24 08:50:17 +02:00
devi
02ae76baf2 add new set order 2023-07-23 16:23:57 +02:00
devi
b5c44383e8 add lxmanga.net 2023-07-23 14:45:46 +02:00
devi
c48ebb937b add Lugnica Scans 2023-07-23 12:29:50 +02:00
devi
49c30d1d26 Merge remote-tracking branch 'origin/master' 2023-07-23 10:16:13 +02:00
devi
cb1a7ece8c fix some source and add 3 source 2023-07-23 10:15:45 +02:00
devi
4d60ae3cb6 fix some source and add 3 source 2023-07-23 10:15:18 +02:00
vianh
fe64620e9f [HentaiVN] Load pages using api 2023-07-23 15:00:36 +07:00
vianh
a42ca9e2da [HentaiVN] New source 2023-07-23 14:06:35 +07:00
506f6f7d30 Merge pull request #217 from davvarrr/master
add source and fix
2023-07-22 13:35:22 +03:00
devi
380861f29e Group sites by language 2023-07-22 12:13:49 +02:00
devi
9afc046d7b Change logic date on mangareader
Change logic domain on mangareader
2023-07-22 11:15:18 +02:00
devi
aec87a9a67 add source and fix japscan 2023-07-22 09:38:56 +02:00
69e0a531df [Madara] Refactoring 2023-07-19 11:49:55 +03:00
c85101991f Merge pull request #214 from davvarrr/master
Add source and create decode url image
2023-07-19 11:14:00 +03:00
devi
99e87fd97a Merge remote-tracking branch 'origin/master' 2023-07-18 20:00:48 +02:00
devi
665ec6b7ed fix japscan 2023-07-18 20:00:33 +02:00
devi
01fbd209c8 Merge branch 'KotatsuApp:master' into master 2023-07-18 19:13:02 +02:00
devi
f0fa8d59a1 add source and create decode url image 2023-07-18 19:12:25 +02:00
a1598fd712 [MangaInUa] Fix parsing chapters and pages 2023-07-18 09:50:55 +03:00
db96a1ff2e [Grouple] Fix tags parsing 2023-07-17 12:20:55 +03:00
833a42e798 Merge pull request #206 from bapeey/tmo
New parser: TuMangaOnline
2023-07-17 12:20:34 +03:00
922c665ef5 Merge pull request #210 from davvarrr/master
Add new source, fix and other
2023-07-17 11:13:13 +03:00
01842e9870 Merge pull request #212 from VietAnh14/master
[BlogTruyen] Update domain, user agent
2023-07-17 11:07:38 +03:00
vianh
a235f6281c [BlogTruyen] Update domain, user agent 2023-07-17 14:52:09 +07:00
devi
c9e7d4f483 add source requested 2023-07-16 20:02:36 +02:00
devi
94496ecc02 fix Asura Scans 2023-07-16 18:51:25 +02:00
devi
5104a8b542 add source request and remove source duplicate 2023-07-16 08:31:50 +02:00
devi
e67beae235 add source request 2023-07-15 21:20:23 +02:00
devi
7c4bffede2 minor fix and add quick source ar 2023-07-15 19:41:28 +02:00
devi
8df620b1cd fix 2023-07-15 19:08:03 +02:00
devi
cbbdd13c8e add new source ( lise below )
fix multiple source. ( lise below )
update madra parser :
-> new option on getListPage for take liste without Ajax
-> add format date
-> change defaut date parterne ( the news is most used by default on sites )
2023-07-15 15:52:07 +02:00
3874fe31d5 [TuMangaOnline] Fixes 2023-07-11 10:58:51 +03:00
a8c086b340 Merge branch 'tmo' of github.com:bapeey/kotatsu-parsers into bapeey-tmo 2023-07-11 10:49:12 +03:00
06a043d290 Refactor Madara parser 2023-07-11 10:36:21 +03:00
5a15014369 Add additional methods to WebClient 2023-07-11 10:08:02 +03:00
7fe1f4b3ff Merge pull request #208 from davvarrr/master
add source and other
2023-07-11 09:56:43 +03:00
devi
0055146264 fix and quick add source request 2023-07-10 17:12:46 +02:00
seew3l
cc26897fb9 Opps 2023-07-10 09:37:35 -05:00
seew3l
46535f82af Apply requested changes 2023-07-10 09:35:26 -05:00
seew3l
23a7b945af Add headers to getPages request 2023-07-09 13:34:15 -05:00
devi
4fb6ff63b9 Merge remote-tracking branch 'origin/master' 2023-07-09 19:56:04 +02:00
devi
c2e90a210c add new source and add other 2023-07-09 19:51:53 +02:00
seew3l
22d59e46c8 Get domain from ConfigKey 2023-07-09 10:18:20 -05:00
seew3l
e462b7df97 Remove unused val 2023-07-09 10:07:25 -05:00
seew3l
af8fb952df Remove comments 2023-07-08 23:28:31 -05:00
seew3l
a3fa04bcb7 Remove comments 2023-07-08 23:27:48 -05:00
seew3l
adf6fea4ac Add TMO 2023-07-08 23:07:25 -05:00
69af4fe586 Merge pull request #203 from davvarrr/master
80 new source
2023-07-06 06:57:00 +03:00
devi
31476a55f9 Fix small bug parse on desc 2023-07-04 21:20:42 +02:00
devi
c833563907 add 10 new source 2023-07-04 18:21:46 +02:00
devi
74a4a3a8b4 And official Manga Origines 2023-07-04 16:01:10 +02:00
devi
141bca517d Add Koitharu optimization, and rename Manga origines 2023-07-04 15:49:50 +02:00
devi
b9eb6363b7 Add 5 new source 2023-07-03 19:31:59 +02:00
devi
c3bed26739 Add 3 new source "fr" 2023-07-03 18:05:30 +02:00
devi
a15d54fe11 Add 10 new source 2023-07-03 17:26:46 +02:00
devi
1998ceac5a Add 3 new sources 2023-07-02 23:01:37 +02:00
devi
4192ffdbfe Add ScantradUnion 2023-07-02 15:31:24 +02:00
devi
5ec7227886 Add ScantradUnion 2023-07-02 15:30:52 +02:00
devi
a9ebc372f6 Add 10 new source 2023-07-02 12:03:07 +02:00
devi
1e98104eca Add new source and add new scrapper for Chapter on theme madara 2023-07-01 21:02:20 +02:00
devi
d05d5fa911 Add new source 2023-07-01 18:33:05 +02:00
07df5a81cf [Chan] Add related manga implementation 2023-07-01 12:27:27 +03:00
9861a9aa6b [Grouple] Add related manga implementation 2023-07-01 12:15:22 +03:00
b65e1c498e Simple related manga search implementation 2023-07-01 11:18:10 +03:00
240562037d Merge pull request #202 from davvarrr/master
rework madara theme and fix many parser
2023-07-01 11:12:55 +03:00
b3f79e77c3 [JapScan] Fix code style 2023-07-01 11:12:24 +03:00
db3c73156e [JapScan] Fix code style 2023-07-01 11:12:15 +03:00
devi
44b3b9b0c1 Merge remote-tracking branch 'origin/master' 2023-06-30 21:22:20 +02:00
devi
ea8c1de44d rework madara theme and fix som parser 2023-06-30 21:22:00 +02:00
devi
8195d0190a remove miss test in local 2023-06-30 09:59:19 +02:00
devi
8fd10eabd1 fix japscan and fix wrong date on 2 source 2023-06-29 20:16:07 +02:00
devi
7b28072ee5 delete table mode for automatic detect mode and chage defaut value for date and list url 2023-06-28 19:59:11 +02:00
devi
b37db4d680 delete table mode for automatic detect mode and chage defaut value for date and list url 2023-06-28 19:58:48 +02:00
92bfc7e9fa [Bentomanga] Fix UserAgent 2023-06-28 13:21:55 +03:00
b3e97760db [DesuMe] Update headers 2023-06-28 12:58:37 +03:00
3895312bfc Refactor MangaReaderParser 2023-06-28 12:31:31 +03:00
06d2976bb5 Batch sources fix 2023-06-28 12:05:41 +03:00
da4566f82f Fix GitHub workflow 2023-06-28 12:01:33 +03:00
b8f0ea4690 Merge pull request #195 from davvarrr/patch-16
Create AiyuMangaScanlation.kt
2023-06-27 12:32:29 +03:00
bb5e49758a Merge pull request #194 from davvarrr/patch-15
54 new source and rework
2023-06-27 12:31:22 +03:00
devi
e617904b62 Update MangaScantrad.kt 2023-06-26 23:25:49 +02:00
devi
b0e455e163 Update ScantradVf.kt 2023-06-26 23:24:09 +02:00
devi
6c31cea155 Update HhentaiFr.kt 2023-06-26 23:23:41 +02:00
devi
9c71d1b57c Update Hentaizone.kt 2023-06-26 23:23:09 +02:00
devi
e4f12149ed fix 2023-06-26 23:22:44 +02:00
devi
f6d264f37d Update MangaScantrad.kt 2023-06-26 23:22:00 +02:00
devi
aa734e5ad3 fix 2023-06-26 23:21:24 +02:00
devi
591291e9d6 fix status 2023-06-26 22:25:35 +02:00
devi
2311a56658 add new parse language for status 2023-06-26 22:00:36 +02:00
514e4473a1 Merge branch 'master' into patch-15 2023-06-26 13:53:57 +03:00
7ccacf4c27 Fix plugins versions 2023-06-26 13:52:49 +03:00
15f226fc73 Update gradle config 2023-06-26 13:29:13 +03:00
1388196288 Update gradle config 2023-06-26 13:27:31 +03:00
fe024cb306 Merge branch 'master' into patch-15 2023-06-26 12:15:48 +03:00
04e7edaa3e Fix GitHub worklflow Java version 2023-06-26 12:11:29 +03:00
devi
87abdf6346 Update MangaReaderParser.kt 2023-06-26 08:08:36 +02:00
devi
c4fad321db Update MangaReaderParser.kt 2023-06-26 07:08:25 +02:00
devi
c17c9d9ccc Update MangaReaderParser.kt
Add long for parse onging / finished
2023-06-26 07:07:01 +02:00
devi
d27c208ab7 Create AiyuMangaScanlation.kt
add aiyumangascanlation.com "es"
2023-06-24 23:15:21 +02:00
devi
90966aa7e4 Update MangaReaderParser.kt
add 54 new source 
rework parseInfoList
Source sorting part language
2023-06-24 23:12:38 +02:00
553a6c709a Reformat code 2023-06-24 16:29:42 +03:00
16698201f4 Merge pull request #191 from davvarrr/patch-12
Create ScantradVf.kt
2023-06-24 13:49:25 +03:00
33f27a8af7 Merge pull request #190 from davvarrr/patch-11
Create MangaScantrad.kt
2023-06-24 13:49:04 +03:00
48ec877ad5 Merge pull request #189 from davvarrr/patch-10
Create Hipercool.kt
2023-06-24 13:48:44 +03:00
54002ce27f Merge pull request #188 from davvarrr/patch-9
Create Hentaiteca.kt
2023-06-24 13:45:47 +03:00
de03d80e10 Merge pull request #186 from davvarrr/patch-7
Create Atlantisscan.kt
2023-06-24 13:45:30 +03:00
60954c38b6 Merge pull request #187 from davvarrr/patch-8
Create FrScan.kt
2023-06-24 13:44:24 +03:00
ba41d5f4ce Merge pull request #184 from davvarrr/patch-5
add french parse date and new source
2023-06-24 13:44:06 +03:00
9b76f42138 Merge pull request #185 from davvarrr/patch-6
Create AstralManga.kt
2023-06-24 13:43:40 +03:00
5a0306b4b5 Merge pull request #192 from davvarrr/patch-13
Create HhentaiFr.kt
2023-06-24 13:43:08 +03:00
b70ddf0a7c Merge pull request #193 from davvarrr/patch-14
Create Hentaizone.kt
2023-06-24 13:42:47 +03:00
devi
f53174258c add Stkissmanga "en"
add Stkissmanga "en"
2023-06-23 19:33:51 +02:00
devi
36706d998e Create Hentaizone.kt 2023-06-23 18:33:44 +02:00
devi
65ec3ca5c6 Update Hipercool.kt
Add parse description
2023-06-23 08:03:58 +02:00
devi
0b0068e113 Update HhentaiFr.kt
large covers are not always present
2023-06-22 22:30:56 +02:00
devi
553777ef08 Create HhentaiFr.kt 2023-06-22 20:59:36 +02:00
devi
3cda7ffc18 Create ScantradVf.kt 2023-06-22 20:52:54 +02:00
devi
48d98548cd Create MangaScantrad.kt 2023-06-22 20:51:49 +02:00
devi
b878824d15 Create Hipercool.kt 2023-06-22 20:50:51 +02:00
devi
16a99d7b8c Create Hentaiteca.kt 2023-06-22 20:49:07 +02:00
devi
c34f250ac9 Create FrScan.kt 2023-06-22 20:48:03 +02:00
devi
e74b276c98 Create Atlantisscan.kt 2023-06-22 20:43:21 +02:00
devi
f0e61daa63 Create AstralManga.kt 2023-06-22 20:42:19 +02:00
devi
34b6f4aa32 add french parse date 2023-06-22 18:49:19 +02:00
c2b79b55f8 Update dependencies 2023-06-22 14:28:47 +03:00
875b08b1fc Merge pull request #182 from davvarrr/patch-3
Create ReaperScansFr.kt
2023-06-22 14:02:50 +03:00
0e3788d943 Merge pull request #181 from davvarrr/patch-2
Add 2 source fr on MangaReaderParser
2023-06-22 14:02:02 +03:00
396508da6f Merge pull request #183 from davvarrr/patch-4
Fix uloaddate and force Enables list
2023-06-22 14:01:03 +03:00
devi
6fc6beb067 Force Enables list
Enables list mode to be forced to load images
2023-06-21 20:03:50 +02:00
devi
939cdf71ad Fix uloaddate
allows you to use the function in madaraparser to fix dates, e.g. "2 hours ago", otherwise the app wouldn't display the chapters.
2023-06-20 21:39:54 +02:00
devi
8d7eabdd71 Create ReaperScansFr.kt
add reaperscans.fr
2023-06-20 19:05:20 +02:00
devi
d28d6b17d8 Update MangaReaderParser.kt
add epsilonscan.fr ( +18 ) "fr"
add legacy-scans.com "fr"
2023-06-20 17:34:28 +02:00
f096ca2ad3 Merge pull request #180 from drsgdfnhtyjktyjt/patch-1
Add Phenixscans.fr
2023-06-20 12:42:39 +03:00
32914eb36d Small fixes 2023-06-20 12:40:15 +03:00
sddsg
b58190ed17 Add Phenixscans.fr
I was finally able to test it at home and it works perfectly.

Could you add it, thank you.
2023-06-19 20:04:32 +02:00
86a82970fc Empty-Commit 2023-06-16 09:54:57 +03:00
9976ea5dfc [Ninemanga] Fix covers issue 2023-06-14 16:12:09 +03:00
f732582d55 [Grouple] Fix chapters parsing 2023-06-13 10:13:54 +03:00
44e28b40d3 Update issue templates 2023-06-02 10:56:20 +03:00
f969e2a8d1 [Grouple] Fix multiple genres search 2023-06-02 10:37:34 +03:00
fa7ea5b16a Fix domain config key 2023-05-27 16:42:43 +03:00
3d7a39cf67 Fix pagination 2023-05-27 12:08:24 +03:00
46f8b8e700 [Ninemanga] Remove redirect check 2023-05-24 17:01:19 +03:00
ebcc6391d6 [Mangalib] Fix nsfw detection 2023-05-23 12:52:59 +03:00
12522ce8d8 [Anibel] Fix parser 2023-05-23 12:15:12 +03:00
cae7073f87 [MangaDex][ComicK] Fix branching again 2023-05-15 15:46:59 +03:00
0366ad0e57 Fix Android compatibility 2023-05-15 11:43:46 +03:00
6b95a8c55e [MangaDex] Fix branching ##172 2023-05-15 10:56:09 +03:00
6183f6edf3 [ComicK] Fix branching #163 #167 2023-05-15 10:21:32 +03:00
83cca4ef12 [KomikTap] fix page size 2023-05-13 15:06:23 +03:00
20ad9d0472 [ALlHentai] New parser 2023-05-13 14:40:15 +03:00
96b9ac36f3 [Bentomanga] Fixes 2023-05-06 17:28:34 +03:00
65170c0fe9 [Mangadex] Fix chapters numbers 2023-05-01 09:03:11 +03:00
3e349d3db3 [Ninemanga] Fix redirects 2023-04-28 17:21:19 +03:00
b2675c3cac Fix @MangaSources annotation 2023-04-28 16:17:47 +03:00
7414cf4a4e Add @JvmField on models fields 2023-04-28 15:43:13 +03:00
f43c482539 [Honeymanga] Refactoring 2023-04-28 15:31:03 +03:00
f0fbbf4333 Merge pull request #169 from CakesTwix/master
[New source] HoneyManga
2023-04-28 15:20:30 +03:00
0d03c68c8b [Manhwaland] Fix list url 2023-04-28 15:20:00 +03:00
CakesTwix
b4a4c86049 [HoneyManga] Impl. filter by genres
and fix cover url
2023-04-26 16:35:42 +03:00
306d46ea93 [Grouple] Fix pages url extraction 2023-04-26 09:24:39 +03:00
5b94badfc2 [MangaRead] Fix list parsing 2023-04-25 08:13:32 +03:00
9adc0c5358 [Bentomanga] New parser 2023-04-24 19:34:29 +03:00
66adc65a96 [JapScan] New parser 2023-04-24 16:51:53 +03:00
8def9b5446 Improve network error handling 2023-04-23 09:20:11 +03:00
e6511061a7 [Grouple] Fix server selecting 2023-04-20 18:07:20 +03:00
CakesTwix
647a694d91 [HoneyManga] throw IllegalArgumentException if search < 3 2023-04-19 20:20:37 +03:00
CakesTwix
e5ececa42d [HoneyManga] Update code, get(), CONTENT_ENCODING 2023-04-19 19:58:25 +03:00
c321faeaa9 Merge pull request #168 from VietAnh14/source/doujindesu
[DoujinDesu] New parser
2023-04-19 19:24:29 +03:00
vianh
3112b7937c [DoujinDesu] Apply suggesions from code review 2023-04-18 23:49:46 +07:00
CakesTwix
a3565abdc3 [HoneyManga] Improve code 2023-04-18 19:35:15 +03:00
CakesTwix
34d2e15302 [New source] HoneyManga 2023-04-18 18:55:43 +03:00
1b6d1456f3 Add support for POST with json body 2023-04-17 08:35:58 +03:00
vianh
b49aee3188 [DoujinDesu] New parser 2023-04-16 22:52:22 +07:00
749f682ef9 [Madara] Fix empty tags 2023-04-15 13:22:33 +03:00
1b269efaa4 Update readme 2023-04-15 13:02:26 +03:00
2db1a8d958 Remove broken parser 2023-04-15 12:59:29 +03:00
ece7d71e09 Update domains 2023-04-15 12:51:29 +03:00
129602ea70 [IsekaiScan] Fix chapters numbers #162 2023-04-15 12:44:17 +03:00
5e3fddb652 [Grouple] Fix relative page urls 2023-04-12 18:35:46 +03:00
2340100999 Fix false nsfw flags on some madara parsers #164 2023-04-06 18:49:19 +03:00
dc0151720e [IsekaiScanEu] Fix chapters order #162 2023-04-04 07:52:24 +03:00
36e88b2c0c [Remanga] Fix chapters numbers 2023-04-04 07:40:47 +03:00
35695904d1 [Remanga] Add Accept header to image requests 2023-03-30 18:52:11 +03:00
a3ffecc00f [IsekaiScan] Fix chapters parsing 2023-03-25 10:14:35 +02:00
cc418570d5 [Remanga] Fix mirror 2023-03-25 07:32:18 +02:00
481af22ae7 [Remanga] Add mirror 2023-03-24 18:02:31 +03:00
96b9dc8b5c [ComicK] Updated API 2023-03-24 16:05:24 +03:00
93f5f70d79 Remove referrer field from page model 2023-03-19 09:28:38 +02:00
fc53b19915 Add toString to models 2023-03-18 18:20:00 +02:00
413f4a2f10 Batch parsers fixes 2023-03-09 07:32:04 +02:00
e8d299782b [Tempestfansub] Update domain #159 2023-03-05 08:09:49 +02:00
489a4ab586 [NeatManga] Fix chapters loading 2023-03-05 08:09:49 +02:00
32fa3a89e2 Merge pull request #152 from OshekharO/patch-1 2023-03-02 09:58:35 +02:00
Saksham Shekher
f2614dd3d7 Fix 2023-02-26 23:03:43 +05:30
1093584202 [Grouple] Add UserAgent config 2023-02-25 18:36:05 +02:00
f62d024342 Fix Manga District & Hentai4free parsers #157 2023-02-25 17:10:41 +02:00
eea87d8607 Fix grouple pages parsing #156 2023-02-25 16:43:06 +02:00
f4c47b5b84 Merge pull request #155 from VietAnh14/source/truyentranhlh
[TruyentranhLH]: New source
2023-02-21 18:28:27 +02:00
vianh
fa78d5dd6f [TruyentranhLH]: New source 2023-02-21 00:47:53 +07:00
779c6664be Merge pull request #154 from OshekharO/patch-2
BatoTo: New Subdomain
2023-02-18 07:18:03 +02:00
Saksham Shekher
38efd7935a BatoTo: New Subdomain 2023-02-18 07:35:24 +05:30
Saksham Shekher
03f021377d Update README.md
some minor grammatical corrections
2023-02-16 23:17:31 +05:30
cf345d2d0c [Grouple] update UserAgent 2023-02-16 19:12:35 +02:00
fb3a3f49dd Merge pull request #150 from OshekharO/patch-1
Add: KumaPoi
2023-02-16 17:59:39 +02:00
f3b731114e [DesuMe] Genres in list 2023-02-16 17:52:27 +02:00
454b24ec88 [Grouple] Selecting the fastest pages server 2023-02-16 17:52:14 +02:00
Saksham Shekher
2c6c64fbe9 Add: ToonHunter 2023-02-14 12:46:10 +05:30
Saksham Shekher
a51d6d6485 Add: Hentai20 2023-02-14 12:37:01 +05:30
Saksham Shekher
0da57c041c Fix 2023-02-14 12:09:40 +05:30
Saksham Shekher
503573f388 Add: KomikLokal 2023-02-14 12:03:18 +05:30
Saksham Shekher
29cd1f1fdf Add: KumaPoi 2023-02-14 11:54:42 +05:30
Saksham Shekher
e51b33c74a Add: Toonily, Komiklab, KomikDewasa, KomiKav (#148)
* Add: Toonily

* Add: KOMIKLAB

* Update MangaReaderParser.kt

* Add: KomikDewasa

* Add: KomiKav
2023-02-09 20:56:04 +03:00
624a3dd991 Merge pull request #146 from KotatsuApp/feature/refactor2
Improve network requests processing
2023-02-08 19:55:51 +02:00
05d705ac03 Remove deprecated "getFavicon" method 2023-02-08 11:08:57 +02:00
1e1756aa17 Refactor web client 2023-02-08 11:01:20 +02:00
91109bf6dc Merge pull request #144 from OshekharO/patch-1
Add: KomikManga & Fixes
2023-02-08 10:50:18 +02:00
Saksham Shekher
7f852e793a Add: BakaMan
Thai Manga Provider
2023-02-08 14:16:15 +05:30
Saksham Shekher
aae3747925 Add: KomikIndo
Closes #145
2023-02-08 08:27:17 +05:30
Saksham Shekher
5e23d52e8a refactor: HachiManga 2023-02-07 10:40:03 +05:30
Saksham Shekher
3bc8013d2f Merge branch 'KotatsuApp:master' into patch-1 2023-02-07 10:20:55 +05:30
00abaea324 [Remanga] Fix headers 2023-02-06 18:57:00 +02:00
d14168e325 Make headers accessible 2023-02-06 18:54:50 +02:00
c3b1556816 Add headers to CloudFlareException 2023-02-06 17:50:44 +02:00
80a2a10e71 [Remanga] Fix headers 2023-02-06 17:30:17 +02:00
Saksham Shekher
029f554d6e Add: DragonTranslation 2023-02-06 14:37:30 +05:30
Saksham Shekher
841d8f05d9 Fix: MangaRosie, PianManga 2023-02-06 14:19:49 +05:30
Saksham Shekher
7253699529 Dead: ReadManwha 2023-02-06 14:18:04 +05:30
Saksham Shekher
5fe546c94d Dead: MangaTX OT
Removed as site is not being updated
2023-02-06 14:16:40 +05:30
Saksham Shekher
31c26f16ac Dead: IsekaiScan
Url changed to isekaiscan.to ( chapter not found in kotatsu )
2023-02-06 14:13:58 +05:30
Saksham Shekher
5eda6f96b4 Dead: ReadManhwa 2023-02-06 14:10:54 +05:30
Saksham Shekher
56a54d5eb4 Fix: NeatManga 2023-02-06 14:10:12 +05:30
Saksham Shekher
7a78a67740 Dead: MangaKik 2023-02-06 14:08:40 +05:30
Saksham Shekher
742533977f Dead: X2Manga 2023-02-06 14:06:39 +05:30
Saksham Shekher
2dedaea0ab Dead: KingManga 2023-02-06 14:05:36 +05:30
Saksham Shekher
cf7f55181f Dead: Manhwa 2023-02-06 14:03:26 +05:30
Saksham Shekher
f68f0009a2 Dead: ManhwaChill 2023-02-06 14:00:07 +05:30
Saksham Shekher
4799f54fd1 Dead: All Top Manga 2023-02-06 13:57:30 +05:30
Saksham Shekher
06ff8df2e4 Dead: Heroxia 2023-02-06 13:52:37 +05:30
Saksham Shekher
9b78944f9b Fix: ManhwaList 2023-02-06 13:49:31 +05:30
Saksham Shekher
fc2781c83a Add: KomikManga 2023-02-06 13:43:02 +05:30
Saksham Shekher
176437531d Add: CosmicScans (#142)
* Add: CosmicScans

* Fix info

* Rem: nsfw

* Mangasu url changed 

Cert issue needs to be fixed
2023-02-05 19:44:48 +03:00
c28e2a72d5 [Remanga] Bypass blocking 2023-02-05 10:09:53 +03:00
7f630184c0 [GroupLe] Fix pages loading 2023-01-29 20:54:52 +03:00
e5a6b82853 [Grouple] Fix chapter link extraction 2023-01-24 09:11:09 +02:00
e1d22b1110 [ExHentai] Option to hide suspicious content 2023-01-24 08:57:19 +02:00
918318252e Fix coroutine cancellation handling in runCatching 2023-01-24 08:03:51 +02:00
e153463c35 [Grouple] Fix authorization recognizing 2023-01-24 07:58:56 +02:00
cf00732023 Misc parsers fixes 2023-01-05 11:26:58 +02:00
ef838068ff [MangahaTachi] Change url (closes #128) 2023-01-01 20:37:04 +03:00
c4acb9725f [Lib] Fix auth part 2 2022-12-31 22:28:46 +03:00
04e219ff69 Remove accidentally duplicated parser 2022-12-31 19:41:36 +03:00
106a85aa80 [Lib] Authorization fixes 2022-12-31 19:03:06 +03:00
ad6e074cce [Asura Scans] New parser 2022-12-29 18:12:22 +03:00
e89f06a35b [KomikTap] New parser (closes #40) (?) 2022-12-29 11:09:25 +03:00
238cc7529a [Asura Scans (tr)] New parser (closes #55) 2022-12-28 23:26:26 +03:00
fb22eb5fac [Tempest Fan Sub] New parser (closes #56) 2022-12-28 23:10:17 +03:00
9ee1c21a67 [ExHentai] Fix pagination 2022-12-26 18:23:15 +02:00
c7c142f4a0 [Remanga] Show bought chapters 2022-12-26 17:46:15 +02:00
efd0394398 [MangaTale] New parser (closes #125) 2022-12-25 12:22:13 +03:00
0091a8280a Merge pull request #122 from VietAnh14/source/heroxia
[MangaReader] Fix chapters order
2022-12-14 08:27:47 +02:00
vianh
f9b0e82473 [MangaReader] Add new source 2022-12-11 00:59:53 +07:00
vianh
63676c21a8 [MangaReader] fix chapters order 2022-12-10 14:55:43 +07:00
add70b4790 [ExHentai] Fix pagination 2022-12-09 18:15:01 +02:00
d18f665556 Merge pull request #121 from VietAnh14/source/heroxia
[MangaReadTheme] Add some new sources using MangaReader theme
2022-12-09 08:18:46 +02:00
vianh
42253e43b3 Add more mangareader theme sources, fix cache tags 2022-12-07 00:25:22 +07:00
vianh
ad500a9db6 [MangaReadTheme] Add some new sources using MangaReader theme 2022-12-06 23:17:31 +07:00
ViAnh
2ab03cb668 [Manhwa18] Fix parse upload date (#115) 2022-11-26 19:37:11 +03:00
ViAnh
b78ddc6166 [NetTruyen] Update domain (#116) 2022-11-26 19:36:41 +03:00
1e49d4095b [Grouple] Fix pages parsing 2022-11-21 09:03:13 +02:00
bf8a1f3db2 [MangaInUa] Fix #104 2022-11-04 17:30:38 +03:00
321cc7529e Merge pull request #96 from VietAnh14/source/manhwa18
[Manhwa18] New source
2022-10-24 19:35:30 +03:00
ViAnh
7907b3126c Apply suggestions from code review
Co-authored-by: Koitharu <nvasya95@gmail.com>
2022-10-24 23:29:53 +07:00
vianh
ccf9af0e51 [Manhwa18] New source 2022-10-24 23:05:13 +07:00
b38b43edc0 Merge pull request #94 from VietAnh14/master
[BlogTruyen] Fix image src attr change
2022-10-24 11:59:42 +03:00
vianh
5d13c55dd5 [BlogTruyen] Fix image src attr change 2022-10-23 20:37:40 +07:00
a1441e7ed7 [MangaDex] Fix large chapters loading error 2022-10-20 09:56:00 +03:00
a62662d116 Fix some madara parsers 2022-10-19 13:37:49 +03:00
5cb953eb86 Small fixes 2022-10-10 16:27:34 +03:00
505ffcf405 Mangalink parser #66 2022-10-10 16:07:14 +03:00
a45f9bfc58 ReaperScansID parser #83 2022-10-10 15:16:40 +03:00
b1990c7918 Fixes 2022-10-01 13:43:09 +03:00
e8b801684f [MangaDex] Fix error on empty description 2022-09-29 14:25:41 +03:00
de0c12c078 Add contribution guidelines 2022-09-14 20:10:18 +03:00
b3a9c5fcda Propagate deprecation from MangaParser to MangaSource 2022-09-07 13:57:00 +03:00
c73d34b04f [Madara] Fix date parsing 2022-09-07 13:45:15 +03:00
583b24854f [Remanga] Omit paid chapters 2022-09-07 13:15:51 +03:00
1c79b29845 Merge branch 'master' of github.com:KotatsuApp/kotatsu-parsers 2022-09-07 13:07:02 +03:00
c348d68983 Add deprecation annotation to some sources 2022-09-04 17:53:29 +03:00
6d7506cc1b [CloneManga] Add source 2022-09-04 17:52:54 +03:00
892 changed files with 30230 additions and 3559 deletions

View File

@@ -6,8 +6,8 @@ end_of_line = lf
indent_style = tab
max_line_length = 120
tab_width = 4
insert_final_newline = false
disabled_rules=no-wildcard-imports,no-unused-imports
insert_final_newline = true
disabled_rules = no-wildcard-imports, no-unused-imports
[{*.kt,*.kts}]
ij_kotlin_allow_trailing_comma = true

View File

@@ -1,65 +1,65 @@
name: 🐞 Issue report
description: Report a source issue in Kotatsu
labels: [bug]
description: Report a source issue with a source
labels: [ bug ]
body:
- type: input
id: source
attributes:
label: Source information
description: |
You can find the source name in navigation drawer.
placeholder: |
Example: "MangaDex"
validations:
required: true
- type: input
id: source
attributes:
label: Source information
description: |
You can find the source name in navigation drawer.
placeholder: |
Example: "MangaDex"
validations:
required: true
- type: textarea
id: reproduce-steps
attributes:
label: Steps to reproduce
description: Provide an example of the issue.
placeholder: |
Example:
1. First step
2. Second step
3. Issue here
validations:
required: false
- type: textarea
id: reproduce-steps
attributes:
label: Steps to reproduce
description: Provide an example of the issue.
placeholder: |
Example:
1. First step
2. Second step
3. Issue here
validations:
required: false
- type: input
id: kotatsu-version
attributes:
label: Kotatsu version
description: |
You can find your Kotatsu version in **Settings → About**.
placeholder: |
Example: "3.3"
validations:
required: true
- type: input
id: kotatsu-version
attributes:
label: Kotatsu version
description: |
You can find your Kotatsu version in **Settings → About**.
placeholder: |
Example: "3.3"
validations:
required: true
- type: input
id: android-version
attributes:
label: Android version
description: |
You can find this somewhere in your Android settings.
placeholder: |
Example: "Android 12"
validations:
required: false
- type: input
id: android-version
attributes:
label: Android version
description: |
You can find this somewhere in your Android settings.
placeholder: |
Example: "Android 12"
validations:
required: false
- type: textarea
id: other-details
attributes:
label: Other details
placeholder: |
Additional details and attachments.
- type: textarea
id: other-details
attributes:
label: Other details
placeholder: |
Additional details and attachments.
- type: checkboxes
id: acknowledgements
attributes:
label: Acknowledgements
options:
- label: I have searched the existing issues and this is a new ticket, **NOT** a duplicate or related to another open issue.
required: true
- type: checkboxes
id: acknowledgements
attributes:
label: Acknowledgements
options:
- label: I have searched the existing issues and this is a new ticket, **NOT** a duplicate or related to another open issue.
required: true

View File

@@ -1,50 +1,53 @@
name: 🌐 Source request
description: Suggest a new source for Kotatsu
labels: [source request]
labels: [ source request ]
body:
- type: input
id: name
attributes:
label: Source name
placeholder: |
Example: "Example Scans"
validations:
required: true
- type: markdown
attributes:
value: Please specify source **name** and **language** in the issue title
- type: input
id: name
attributes:
label: Source name
placeholder: |
Example: "Example Scans"
validations:
required: true
- type: input
id: link
attributes:
label: Source link
placeholder: |
Example: "https://example.org"
validations:
required: true
- type: input
id: link
attributes:
label: Source link
placeholder: |
Example: "https://example.org"
validations:
required: true
- type: input
id: language
attributes:
label: Language
placeholder: |
Example: "English"
validations:
required: true
- type: input
id: language
attributes:
label: Language
placeholder: |
Example: "English"
validations:
required: true
- type: textarea
id: other-details
attributes:
label: Other details
placeholder: |
Additional details and attachments.
- type: textarea
id: other-details
attributes:
label: Other details
placeholder: |
Additional details and attachments.
- type: checkboxes
id: acknowledgements
attributes:
label: Acknowledgements
options:
- label: I have searched the existing issues and this is a new ticket, **NOT** a duplicate or related to another open issue.
required: true
- label: I have checked that the source does not already exist on the app.
required: true
- label: I have checked that the source does not already exist by searching the [GitHub repository](https://github.com/KotatsuApp/kotatsu-parsers) and verified it does not appear in the code base.
required: true
- type: checkboxes
id: acknowledgements
attributes:
label: Acknowledgements
options:
- label: I have searched the existing issues and this is a new ticket, **NOT** a duplicate or related to another open issue.
required: true
- label: I have checked that the source does not already exist on the app.
required: true
- label: I have checked that the source does not already exist by searching the [GitHub repository](https://github.com/KotatsuApp/kotatsu-parsers) and verified it does not appear in the code base.
required: true

View File

@@ -1,27 +1,27 @@
name: Parsers test
on:
workflow_dispatch:
pull_request:
paths:
- 'src/main/kotlin/org/koitharu/kotatsu/parsers/site/*'
workflow_dispatch:
pull_request:
paths:
- 'src/main/kotlin/org/koitharu/kotatsu/parsers/site/**'
permissions:
contents: read
contents: read
jobs:
build-and-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-java@v3
with:
java-version: '11'
distribution: 'temurin'
cache: 'gradle'
- run: ./gradlew :test --tests "org.koitharu.kotatsu.parsers.MangaParserTest" || true
- run: ./gradlew generateTestsReport
- uses: actions/upload-artifact@v3
with:
name: Report
path: build/test-results-html/TEST-org.koitharu.kotatsu.parsers.MangaParserTest.htm
build-and-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-java@v3
with:
java-version: '11'
distribution: 'temurin'
cache: 'gradle'
- run: ./gradlew :test --tests "org.koitharu.kotatsu.parsers.MangaParserTest" || true
- run: ./gradlew generateTestsReport
- uses: actions/upload-artifact@v3
with:
name: Report
path: build/test-results-html/TEST-org.koitharu.kotatsu.parsers.MangaParserTest.htm

2
.idea/kotlinc.xml generated
View File

@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="KotlinJpsPluginSettings">
<option name="version" value="1.6.21" />
<option name="version" value="1.9.0" />
</component>
</project>

91
CONTRIBUTING.md Normal file
View File

@@ -0,0 +1,91 @@
# Contributing
The following is guide for creating a Kotatsu parsers. Thanks for taking the time to contribute!
## Prerequisites
Before you start, please note that the ability to use following technologies is **required**.
- Basic [Android development](https://developer.android.com/)
- [Kotlin](https://kotlinlang.org/)
- Web scraping ([JSoup](https://jsoup.org/)) or JSON API
### Tools
- [Android Studio](https://developer.android.com/studio)
- [IntelliJ IDEA](https://www.jetbrains.com/idea/) (Community edition is enough)
- Android device (or emulator)
Kotatsu parsers is not a part of Android application, but you can easily develop and test it directly inside an Android
application project and relocate it to the library project when done.
### Before you start
First, take a look at `kotatsu-parsers` project structure. Each parser is a single class that
extends `MangaParser` class and have a `MangaSourceParser` annotation.
Also pay attention on extensions in `util` package. For example, extensions from `Jsoup` file
should be used instead of existing JSoup functions because they have better nullability support
and improved error messages.
## Writing your parser
So, you want to create a parser, that will provide access to manga from a website.
First, you should explore a website for API availability.
If it does not contain any documentation about
API, [explore network requests](https://firefox-source-docs.mozilla.org/devtools-user/):
some websites use ajax.
- [Example](https://github.com/KotatsuApp/kotatsu-parsers/blob/master/src/main/kotlin/org/koitharu/kotatsu/parsers/site/ru/DesuMeParser.kt)
of Json API usage.
- [Example](https://github.com/KotatsuApp/kotatsu-parsers/blob/master/src/main/kotlin/org/koitharu/kotatsu/parsers/site/be/AnibelParser.kt)
of GraphQL API usage
- [Example](https://github.com/KotatsuApp/kotatsu-parsers/blob/master/src/main/kotlin/org/koitharu/kotatsu/parsers/site/en/MangaTownParser.kt)
of pure HTML parsing.
If website is based on some engine it is rationally to use common base class for this one (for example, Madara wordress
theme
and the `MadaraParser` class)
### Parser class skeleton
Parser class must have exactly one primary constructor parameter of type `MangaLoaderContext` and have an
`MangaSourceParser` annotation that provides internal name, title and language of a manga source.
All functions in `MangaParser` class are documented. Pay attention to some peculiarities:
- Never hardcode domain. Specify default domain in `configKeyDomain` field and obtain an actual one using `getDomain()`.
- All ids must be unique and domain-independent. Use `generateUid` functions with relative url or some internal id which
is unique across the manga source.
- `sortOrders` set should not be empty. If your source is not support sorting, specify one most relevance value.
- If you cannot obtain direct links to pages images inside `getPages` method, it is ok to use an intermediate url
as `Page.url` and fetch a direct link at `getPageUrl` function.
- You can use _asserts_ to check some optional fields. For example. `Manga.author` field is not required, but if your
source provide such information, add `assert(it != null)`. This will not have any effect on production but help to
find issues during unit testing.
- If your source website (or it's api) uses pages for pagination instead of offset you should extend `PagedMangaParser`
instead of `MangaParser`.
- Your parser may also implement the `Interceptor` interface for additional manipulation of all network requests and/or
responses, including image loading.
## Development process
During the development it is recommended (but not necessary) to write it directly
in the Kotatsu android application project. You can use `core.parser.DummyParser` class as a sandbox. `Dummy` manga
source is available in debug Kotatsu build.
Once parser is ready you can relocate your code into `kotatsu-parsers` library project in a `site` package and create a
Pull Request.
### Testing
It is recommended to run unit tests before submitting a PR.
- Temporary modify the `MangaSources` annotation class: specify your parser(s) name(s) and change mode
to `EnumSource.Mode.INCLUDE`
- Run the `MangaParserTest` (`gradlew :test --tests "org.koitharu.kotatsu.parsers.MangaParserTest"`)
- Optionally, you can run the `generateTestsReport` gradle task to get a pretty readable html report from test results.
## Help
If you need a help or have some questions, ask a community in our [Telegram chat](https://t.me/kotatsuapp)
or [Discord server](https://discord.gg/NNJ5RgVBC5).

View File

@@ -1,12 +1,12 @@
# Kotatsu parsers
Library that provides manga sources.
This library provides manga sources.
[![](https://jitpack.io/v/KotatsuApp/kotatsu-parsers.svg)](https://jitpack.io/#KotatsuApp/kotatsu-parsers) ![Kotlin](https://img.shields.io/github/languages/top/KotatsuApp/kotatsu-parsers) ![License](https://img.shields.io/github/license/KotatsuApp/Kotatsu) [![Discord](https://img.shields.io/discord/898363402467045416?color=5865f2&label=discord)](https://discord.gg/NNJ5RgVBC5)
[![](https://jitpack.io/v/KotatsuApp/kotatsu-parsers.svg)](https://jitpack.io/#KotatsuApp/kotatsu-parsers) ![Kotlin](https://img.shields.io/github/languages/top/KotatsuApp/kotatsu-parsers) ![License](https://img.shields.io/github/license/KotatsuApp/Kotatsu) [![Telegram](https://img.shields.io/badge/chat-telegram-60ACFF)](https://t.me/kotatsuapp) [![Discord](https://img.shields.io/discord/898363402467045416?color=5865f2&label=discord)](https://discord.gg/NNJ5RgVBC5)
### Usage
## Usage
1. Add it in your root build.gradle at the end of repositories:
1. Add it to your root build.gradle at the end of repositories:
```groovy
allprojects {
@@ -35,17 +35,27 @@ Library that provides manga sources.
}
```
See for versions at [JitPack](https://jitpack.io/#KotatsuApp/kotatsu-parsers)
Versions are available on [JitPack](https://jitpack.io/#KotatsuApp/kotatsu-parsers)
3. Usage in code
```kotlin
val parser = MangaSource.MANGADEX.newParser(mangaLoaderContext)
val parser = mangaLoaderContext.newParserInstance(MangaSource.MANGADEX)
```
`mangaLoaderContext` is an implementation of the `MangaLoaderContext` class.
See [Android](https://github.com/KotatsuApp/Kotatsu/blob/devel/app/src/main/java/org/koitharu/kotatsu/core/parser/MangaLoaderContextImpl.kt)
See examples
of [Android](https://github.com/KotatsuApp/Kotatsu/blob/devel/app/src/main/java/org/koitharu/kotatsu/core/parser/MangaLoaderContextImpl.kt)
and [Non-Android](https://github.com/KotatsuApp/kotatsu-dl/blob/master/src/main/kotlin/org/koitharu/kotatsu_dl/env/MangaLoaderContextImpl.kt)
implementation examples.
implementation.
Note that the `MangaSource.LOCAL` and `MangaSource.DUMMY` parsers cannot be instantiated.
Note that the `MangaSource.LOCAL` and `MangaSource.DUMMY` parsers cannot be instantiated.
## Contribution
See [CONTRIBUTING.md](./CONTRIBUTING.md) for the guidelines.
## DMCA disclaimer
The developers of this application have no affiliation with the content available in the app. It is collected from
sources freely available through any web browser.

View File

@@ -2,8 +2,8 @@ import tasks.ReportGenerateTask
plugins {
id 'java-library'
id 'org.jetbrains.kotlin.jvm'
id 'com.google.devtools.ksp'
id 'org.jetbrains.kotlin.jvm' version '1.9.20'
id 'com.google.devtools.ksp' version '1.9.20-1.0.14'
id 'maven-publish'
}
@@ -16,28 +16,27 @@ test {
compileKotlin {
kotlinOptions {
jvmTarget = '1.8'
freeCompilerArgs += [
'-opt-in=kotlin.RequiresOptIn',
'-opt-in=kotlin.contracts.ExperimentalContracts',
'-opt-in=kotlinx.coroutines.ExperimentalCoroutinesApi',
'-opt-in=org.koitharu.kotatsu.parsers.InternalParsersApi',
'-opt-in=kotlin.RequiresOptIn',
'-opt-in=kotlin.contracts.ExperimentalContracts',
'-opt-in=kotlinx.coroutines.ExperimentalCoroutinesApi',
'-opt-in=org.koitharu.kotatsu.parsers.InternalParsersApi',
]
}
}
compileTestKotlin {
kotlinOptions {
jvmTarget = '1.8'
freeCompilerArgs += [
'-opt-in=kotlin.RequiresOptIn',
'-opt-in=kotlinx.coroutines.ExperimentalCoroutinesApi',
'-opt-in=org.koitharu.kotatsu.parsers.InternalParsersApi',
'-opt-in=kotlin.RequiresOptIn',
'-opt-in=kotlinx.coroutines.ExperimentalCoroutinesApi',
'-opt-in=org.koitharu.kotatsu.parsers.InternalParsersApi',
]
}
}
kotlin {
jvmToolchain(11)
sourceSets {
main.kotlin.srcDirs += 'build/generated/ksp/main/kotlin'
}
@@ -54,20 +53,20 @@ afterEvaluate {
}
dependencies {
implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-core:1.6.4'
implementation 'com.squareup.okhttp3:okhttp:4.10.0'
implementation 'com.squareup.okio:okio:3.2.0'
api 'org.jsoup:jsoup:1.15.2'
implementation 'org.json:json:20220320'
implementation 'androidx.collection:collection-ktx:1.2.0'
implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-core:1.7.3'
implementation 'com.squareup.okhttp3:okhttp:4.12.0'
implementation 'com.squareup.okio:okio:3.6.0'
api 'org.jsoup:jsoup:1.16.2'
implementation 'org.json:json:20231013'
implementation 'androidx.collection:collection-ktx:1.3.0'
ksp project(':kotatsu-parsers-ksp')
testImplementation 'org.junit.jupiter:junit-jupiter-api:5.9.0'
testImplementation 'org.junit.jupiter:junit-jupiter-engine:5.9.0'
testImplementation 'org.junit.jupiter:junit-jupiter-params:5.9.0'
testImplementation 'org.jetbrains.kotlinx:kotlinx-coroutines-test:1.6.4'
testImplementation 'org.junit.jupiter:junit-jupiter-api:5.10.0'
testImplementation 'org.junit.jupiter:junit-jupiter-engine:5.10.0'
testImplementation 'org.junit.jupiter:junit-jupiter-params:5.10.0'
testImplementation 'org.jetbrains.kotlinx:kotlinx-coroutines-test:1.7.3'
testImplementation 'io.webfolder:quickjs:1.1.0'
}
task generateTestsReport(type: ReportGenerateTask)
tasks.register('generateTestsReport', ReportGenerateTask)

View File

@@ -1,5 +1,5 @@
plugins {
id('org.jetbrains.kotlin.jvm') version '1.6.21'
id 'org.jetbrains.kotlin.jvm' version '1.8.22'
}
repositories {
mavenCentral()
@@ -8,6 +8,6 @@ repositories {
dependencies {
implementation gradleApi()
implementation 'org.simpleframework:simple-xml:2.7.1'
implementation 'com.soywiz.korlibs.korte:korte-jvm:3.0.0-Beta5'
implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-core:1.6.3'
}
implementation 'com.soywiz.korlibs.korte:korte-jvm:4.0.9'
implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-core:1.7.3'
}

View File

@@ -1,6 +1,6 @@
package tasks
import com.soywiz.korte.Template
import korlibs.template.Template
import kotlinx.coroutines.runBlocking
import org.gradle.api.DefaultTask
import org.gradle.api.tasks.TaskAction
@@ -42,10 +42,13 @@ open class ReportGenerateTask : DefaultTask() {
val results = LinkedHashMap<String, LinkedHashMap<String, TestCase>>()
val tests = LinkedHashSet<String>()
for (case in testSuite.testCases) {
if (!case.isValid()) {
continue
}
tests.add(case.testName)
val map = results.getOrPut(case.source) { LinkedHashMap() }
val oldValue = map.put(case.testName, case)
check(oldValue == null)
check(oldValue == null) { "Check failed: $oldValue" }
}
val failPercent = (testSuite.failures.toDouble() / testSuite.tests * 100.0).roundToInt()

View File

@@ -19,14 +19,16 @@ class TestCase {
var failure: Failure? = null
val index by lazy {
name.split('|')[0].toInt()
name.split('|').getOrNull(0)?.toIntOrNull() ?: 0
}
val testName by lazy {
name.split('|')[1]
name.split('|').getOrNull(1).orEmpty()
}
val source by lazy {
name.split('|')[2]
name.split('|').getOrNull(2).orEmpty()
}
}
fun isValid() = name.count { it == '|' } == 2
}

View File

@@ -2,13 +2,15 @@
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta content="width=device-width, initial-scale=1" name="viewport">
<title>{{ testSuite.name }}</title>
<link href="https://cdn.jsdelivr.net/npm/bootstrap@5.2.0-beta1/dist/css/bootstrap.min.css" rel="stylesheet"
integrity="sha384-0evHe/X+R7YkIZDRvuzKMRqM+OrBnVFBL6DOitfPri4tjfHxaWutUpFmBp4vmVor" crossorigin="anonymous">
<script src="https://cdn.jsdelivr.net/npm/bootstrap@5.2.0-beta1/dist/js/bootstrap.bundle.min.js"
integrity="sha384-pprn3073KE6tl6bjs2QrFaJGz5/SUsLqktiwsUTF55Jfv3qYSDhgCecCxMW52nD2"
crossorigin="anonymous"></script>
<!-- CSS only -->
<link crossorigin="anonymous" href="https://cdn.jsdelivr.net/npm/bootstrap@5.2.2/dist/css/bootstrap.min.css"
integrity="sha384-Zenh87qX5JnK2Jl0vWa8Ck2rdkQ2Bzep5IDxbcnCeuOxjzrPF/et3URy9Bv1WTRi" rel="stylesheet">
<!-- JavaScript Bundle with Popper -->
<script crossorigin="anonymous"
integrity="sha384-OERcA2EqjJCMA+/3y+gxIOqMEjwtxJY7qPCqsdltbNJuaOe923+mo//f6V8Qbsw3"
src="https://cdn.jsdelivr.net/npm/bootstrap@5.2.2/dist/js/bootstrap.bundle.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/feather-icons/dist/feather.min.js"></script>
</head>
<body class="py-4">
@@ -25,67 +27,80 @@
{{ testSuite.errors }} ({{ error_percent }}%)
</div>
</div>
<div class="table-responsive mt-4">
<table class="table table-hover">
<thead>
<tr>
<th scope="col">Source</th>
{% for test in tests %}
<th scope="col" style="min-width: 5em;">{{ test }}</th>
{% endfor %}
</tr>
</thead>
{% for name, cases in results %}
<tr>
<th scope="row">{{ name }}</th>
{% for test in tests %}
{% set case = cases[test] %}
{% if case.failure == null %}
<td class="table-success text-center">
<i data-feather="check"></i>
</td>
{% else %}
{% if case.failure.type == 'java.lang.AssertionError' %}
<td class="table-warning text-center" style="cursor: pointer;"
data-bs-toggle="modal" data-bs-target="#failure_{{ case.hashCode }}">
<i data-feather="alert-triangle"></i>
</td>
{% else %}
<td class="table-danger text-center" style="cursor: pointer;"
data-bs-toggle="modal" data-bs-target="#failure_{{ case.hashCode }}">
<i data-feather="x"></i>
</td>
{% endif %}
<!--suppress HtmlUnknownTag -->
<div class="modal fade" id="failure_{{ case.hashCode }}" tabindex="-1">
<div class="modal-dialog modal-lg modal-dialog-scrollable">
<div class="modal-content">
<div class="modal-header">
<h5 class="modal-title">{{ case.testName }} failed</h5>
<button type="button" class="btn-close" data-bs-dismiss="modal"
aria-label="Close"></button>
</div>
<div class="modal-body font-monospace lh-sm bg-light" style="font-size: 0.7em;">
{{ case.failure.textHtml()|raw }}
</div>
<div class="modal-footer">
<button type="button" class="btn btn-secondary" data-bs-dismiss="modal">Close</button>
</div>
<table class="table table-hover">
<thead class="sticky-top bg-body">
<tr>
<th scope="col">Source</th>
{% for test in tests %}
<th class="text-center" scope="col" style="min-width: 5em;">{{ test }}</th>
{% endfor %}
</tr>
</thead>
{% for name, cases in results %}
<tr>
<th scope="row">{{ name }}</th>
{% for test in tests %}
{% set case = cases[test] %}
{% if case.failure == null %}
<td class="table-success text-center">
<i data-feather="check"></i>
</td>
{% else %}
{% if case.failure.type == 'java.lang.AssertionError' %}
<td class="table-warning text-center" data-bs-target="#failure_{{ case.hashCode }}"
data-bs-toggle="modal" style="cursor: pointer;">
<i data-feather="alert-triangle"></i>
</td>
{% elseif case.failure.type == 'java.net.SocketTimeoutException' or case.failure.type ==
'java.net.UnknownHostException' %}
<td class="table-secondary text-center" data-bs-target="#failure_{{ case.hashCode }}"
data-bs-toggle="modal" style="cursor: pointer;">
<i data-feather="power"></i>
</td>
{% elseif case.failure.type == 'org.koitharu.kotatsu.parsers.CloudFlareProtectedException' %}
<td class="table-secondary text-center" data-bs-target="#failure_{{ case.hashCode }}"
data-bs-toggle="modal" style="cursor: pointer;">
<i data-feather="shield"></i>
</td>
{% elseif case.failure.type == 'org.koitharu.kotatsu.parsers.exception.AuthRequiredException' %}
<td class="table-secondary text-center" data-bs-target="#failure_{{ case.hashCode }}"
data-bs-toggle="modal" style="cursor: pointer;">
<i data-feather="user-x"></i>
</td>
{% else %}
<td class="table-danger text-center" data-bs-target="#failure_{{ case.hashCode }}"
data-bs-toggle="modal" style="cursor: pointer;">
<i data-feather="x"></i>
</td>
{% endif %}
<!--suppress HtmlUnknownTag -->
<div class="modal fade" id="failure_{{ case.hashCode }}" tabindex="-1">
<div class="modal-dialog modal-lg modal-dialog-scrollable">
<div class="modal-content">
<div class="modal-header">
<h5 class="modal-title">{{ case.testName }} failed</h5>
<button aria-label="Close" class="btn-close" data-bs-dismiss="modal"
type="button"></button>
</div>
<div class="modal-body font-monospace lh-sm bg-light" style="font-size: 0.7em;">
{{ case.failure.textHtml()|raw }}
</div>
<div class="modal-footer">
<button class="btn btn-secondary" data-bs-dismiss="modal" type="button">Close</button>
</div>
</div>
</div>
{% endif %}
{% endfor %}
</tr>
</div>
{% endif %}
{% endfor %}
</table>
</div>
</tr>
{% endfor %}
</table>
</div>
<script>
feather.replace()
</script>
</body>
</html>
</html>

View File

@@ -1 +1,2 @@
kotlin.code.style=official
org.gradle.jvmargs=-Xmx4g -XX:MaxMetaspaceSize=4096m

View File

@@ -1,5 +1,5 @@
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-7.3.3-bin.zip
distributionUrl=https\://services.gradle.org/distributions/gradle-8.2-bin.zip
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists

View File

@@ -3,5 +3,5 @@ plugins {
}
dependencies {
implementation 'com.google.devtools.ksp:symbol-processing-api:1.6.21-1.0.5'
}
implementation 'com.google.devtools.ksp:symbol-processing-api:1.9.20-1.0.14'
}

View File

@@ -69,10 +69,14 @@ class ParserProcessor(
import org.koitharu.kotatsu.parsers.model.MangaSource
@Suppress("DEPRECATION")
@InternalParsersApi
@Deprecated("", replaceWith = ReplaceWith("context.newParserInstance(this)"))
fun MangaSource.newParser(context: MangaLoaderContext): MangaParser = when (this) {
""".trimIndent(),
)
//language=kotlin
sourcesWriter?.write(
"""
package org.koitharu.kotatsu.parsers.model
@@ -80,8 +84,9 @@ class ParserProcessor(
enum class MangaSource(
val title: String,
val locale: String?,
val contentType: ContentType,
) {
LOCAL("Local", null),
LOCAL("Local", null, ContentType.OTHER),
""".trimIndent(),
)
@@ -104,7 +109,7 @@ class ParserProcessor(
)
sourcesWriter?.write(
"""
DUMMY("Dummy", null),
DUMMY("Dummy", null, ContentType.OTHER),
;
}
""".trimIndent(),
@@ -116,14 +121,18 @@ class ParserProcessor(
private val factoryWriter: Writer?,
) : KSVisitorVoid() {
private val titles = HashMap<String, String>()
override fun visitClassDeclaration(classDeclaration: KSClassDeclaration, data: Unit) {
if (classDeclaration.classKind != ClassKind.CLASS || classDeclaration.isAbstract()) {
logger.error("Only non-abstract can be annotated with @MangaSourceParser", classDeclaration)
}
val annotation = classDeclaration.annotations.single { it.shortName.asString() == "MangaSourceParser" }
val deprecation = classDeclaration.annotations.singleOrNull { it.shortName.asString() == "Deprecated" }
val name = annotation.arguments.single { it.name?.asString() == "name" }.value as String
val title = annotation.arguments.single { it.name?.asString() == "title" }.value as String
val locale = annotation.arguments.single { it.name?.asString() == "locale" }.value as String
val type = annotation.arguments.single { it.name?.asString() == "type" }.value
val localeString = if (locale.isEmpty()) "null" else "\"$locale\""
val localeObj = if (locale.isEmpty()) null else Locale(locale)
val localeTitle = localeObj?.getDisplayLanguage(localeObj)
@@ -142,11 +151,21 @@ class ParserProcessor(
classDeclaration,
)
}
val className = checkNotNull(classDeclaration.qualifiedName?.asString()) { "Class name is null" }
val prevTitleClass = titles.put(title, className)
if (prevTitleClass != null) {
logger.warn("Source title duplication: \"$title\" is assigned to both $prevTitleClass and $className")
}
val className = classDeclaration.qualifiedName?.asString()
factoryWriter?.write("\tMangaSource.$name -> $className(context)\n")
val deprecationString = if (deprecation != null) {
val reason = deprecation.arguments
.find { it.name?.asString() == "message" }?.value?.toString() ?: "Unknown reason"
"@Deprecated(\"$reason\") "
} else ""
val localeComment = localeTitle?.toTitleCase(localeObj)?.let { " /* $it */" }.orEmpty()
sourcesWriter?.write("\t$name(\"$title\", $localeString$localeComment),\n")
sourcesWriter?.write("\t$deprecationString$name(\"$title\", $localeString$localeComment, ContentType.$type),\n")
}
}
}
}

View File

@@ -1,19 +1,16 @@
pluginManagement {
plugins {
id 'com.google.devtools.ksp' version '1.6.21-1.0.5'
id 'org.jetbrains.kotlin.jvm' version '1.6.21'
}
repositories {
gradlePluginPortal()
google()
mavenCentral()
gradlePluginPortal()
}
}
dependencyResolutionManagement {
repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS)
repositories {
google()
mavenCentral()
gradlePluginPortal()
}
}

View File

@@ -11,4 +11,4 @@ package org.koitharu.kotatsu.parsers
@SinceKotlin("1.3")
@RequiresOptIn
@MustBeDocumented
annotation class InternalParsersApi()
annotation class InternalParsersApi

View File

@@ -1,136 +1,19 @@
package org.koitharu.kotatsu.parsers
import okhttp3.*
import okhttp3.HttpUrl.Companion.toHttpUrl
import okhttp3.MediaType.Companion.toMediaType
import okhttp3.RequestBody.Companion.toRequestBody
import org.json.JSONObject
import org.jsoup.HttpStatusException
import okhttp3.CookieJar
import okhttp3.OkHttpClient
import org.koitharu.kotatsu.parsers.config.MangaSourceConfig
import org.koitharu.kotatsu.parsers.exception.GraphQLException
import org.koitharu.kotatsu.parsers.exception.NotFoundException
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.util.await
import org.koitharu.kotatsu.parsers.util.parseJson
import java.util.*
abstract class MangaLoaderContext {
protected abstract val httpClient: OkHttpClient
abstract val httpClient: OkHttpClient
abstract val cookieJar: CookieJar
/**
* Do a GET http request to specific url
* @param url
* @param headers an additional headers for request, may be null
*/
suspend fun httpGet(url: HttpUrl, headers: Headers? = null): Response {
val request = Request.Builder()
.get()
.url(url)
if (headers != null) {
request.headers(headers)
}
return httpClient.newCall(request.build()).await().ensureSuccess()
}
suspend fun httpGet(url: String, headers: Headers? = null): Response {
return httpGet(url.toHttpUrl(), headers)
}
/**
* Do a HEAD http request to specific url
* @param url
* @param headers an additional headers for request, may be null
*/
suspend fun httpHead(url: String, headers: Headers? = null): Response {
val request = Request.Builder()
.head()
.url(url)
if (headers != null) {
request.headers(headers)
}
return httpClient.newCall(request.build()).await().ensureSuccess()
}
/**
* Do a POST http request to specific url with `multipart/form-data` payload
* @param url
* @param form payload as key=>value map
* @param headers an additional headers for request, may be null
*/
suspend fun httpPost(
url: String,
form: Map<String, String>,
headers: Headers? = null,
): Response {
val body = FormBody.Builder()
form.forEach { (k, v) ->
body.addEncoded(k, v)
}
val request = Request.Builder()
.post(body.build())
.url(url)
if (headers != null) {
request.headers(headers)
}
return httpClient.newCall(request.build()).await().ensureSuccess()
}
/**
* Do a POST http request to specific url with `multipart/form-data` payload
* @param url
* @param payload payload as `key=value` string with `&` separator
* @param headers an additional headers for request, may be null
*/
suspend fun httpPost(
url: String,
payload: String,
headers: Headers?,
): Response {
val body = FormBody.Builder()
payload.split('&').forEach {
val pos = it.indexOf('=')
if (pos != -1) {
val k = it.substring(0, pos)
val v = it.substring(pos + 1)
body.addEncoded(k, v)
}
}
val request = Request.Builder()
.post(body.build())
.url(url)
if (headers != null) {
request.headers(headers)
}
return httpClient.newCall(request.build()).await().ensureSuccess()
}
/**
* Do a GraphQL request to specific url
* @param endpoint an url
* @param query GraphQL request payload
*/
suspend fun graphQLQuery(endpoint: String, query: String): JSONObject {
val body = JSONObject()
body.put("operationName", null as Any?)
body.put("variables", JSONObject())
body.put("query", "{$query}")
val mediaType = "application/json; charset=utf-8".toMediaType()
val requestBody = body.toString().toRequestBody(mediaType)
val request = Request.Builder()
.post(requestBody)
.url(endpoint)
val json = httpClient.newCall(request.build()).await().ensureSuccess().parseJson()
json.optJSONArray("errors")?.let {
if (it.length() != 0) {
throw GraphQLException(it)
}
}
return json
}
@Suppress("DEPRECATION")
fun newParserInstance(source: MangaSource): MangaParser = source.newParser(this)
open fun encodeBase64(data: ByteArray): String = Base64.getEncoder().encodeToString(data)
@@ -146,21 +29,4 @@ abstract class MangaLoaderContext {
abstract suspend fun evaluateJs(script: String): String?
abstract fun getConfig(source: MangaSource): MangaSourceConfig
private fun Response.ensureSuccess(): Response {
val exception: Exception? = when (code) { // Catch some error codes, not all
404 -> NotFoundException(message, request.url.toString())
in 500..599 -> HttpStatusException(message, code, request.url.toString())
else -> null
}
if (exception != null) {
runCatching {
close()
}.onFailure {
exception.addSuppressed(it)
}
throw exception
}
return this
}
}
}

View File

@@ -1,63 +1,96 @@
package org.koitharu.kotatsu.parsers
import androidx.annotation.CallSuper
import androidx.annotation.VisibleForTesting
import okhttp3.Headers
import okhttp3.HttpUrl
import org.jsoup.nodes.Element
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.exception.ParseException
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.network.OkHttpWebClient
import org.koitharu.kotatsu.parsers.network.UserAgents
import org.koitharu.kotatsu.parsers.network.WebClient
import org.koitharu.kotatsu.parsers.util.FaviconParser
import org.koitharu.kotatsu.parsers.util.RelatedMangaFinder
import org.koitharu.kotatsu.parsers.util.domain
import org.koitharu.kotatsu.parsers.util.toAbsoluteUrl
import java.util.*
abstract class MangaParser @InternalParsersApi constructor(val source: MangaSource) {
protected abstract val context: MangaLoaderContext
abstract class MangaParser @InternalParsersApi constructor(
@property:InternalParsersApi val context: MangaLoaderContext,
val source: MangaSource,
) {
/**
* Supported [SortOrder] variants. Must not be empty.
*
* For better performance use [EnumSet] for more than one item.
*/
abstract val sortOrders: Set<SortOrder>
abstract val availableSortOrders: Set<SortOrder>
/**
* Supported [MangaState] variants for filtering. May be empty.
*
* For better performance use [EnumSet] for more than one item.
*/
open val availableStates: Set<MangaState>
get() = emptySet()
/**
* Whether parser supports filtering by more than one tag
*/
open val isMultipleTagsSupported: Boolean = true
@Deprecated(
message = "Use availableSortOrders instead",
replaceWith = ReplaceWith("availableSortOrders"),
)
val sortOrders: Set<SortOrder>
get() = availableSortOrders
val config by lazy { context.getConfig(source) }
val sourceLocale: Locale?
get() = source.locale?.let { Locale(it) }
open val sourceLocale: Locale
get() = source.locale?.let { Locale(it) } ?: Locale.ROOT
val isNsfwSource = source.contentType == ContentType.HENTAI
/**
* Provide default domain and available alternatives, if any.
*
* Never hardcode domain in requests, use [getDomain] instead.
* Never hardcode domain in requests, use [domain] instead.
*/
protected abstract val configKeyDomain: ConfigKey.Domain
@InternalParsersApi
abstract val configKeyDomain: ConfigKey.Domain
@VisibleForTesting(otherwise = VisibleForTesting.PROTECTED)
internal open val headers: Headers? = null
open val headers: Headers = Headers.Builder()
.add("User-Agent", UserAgents.CHROME_MOBILE)
.build()
/**
* Used as fallback if value of `sortOrder` passed to [getList] is null
*/
protected open val defaultSortOrder: SortOrder
get() {
val supported = sortOrders
return SortOrder.values().first { it in supported }
val supported = availableSortOrders
return SortOrder.entries.first { it in supported }
}
@JvmField
protected val webClient: WebClient = OkHttpWebClient(context.httpClient, source)
/**
* Parse list of manga by specified criteria
*
* @param offset starting from 0 and used for pagination.
* Note than passed value may not be divisible by internal page size, so you should adjust it manually.
* @param query search query, may be null or empty if no search needed
* @param tags genres for filtering, values from [getTags] and [Manga.tags]. May be null or empty
* @param sortOrder one of [sortOrders] or null for default value
* @param tags genres for filtering, values from [getAvailableTags] and [Manga.tags]. May be null or empty
* @param sortOrder one of [availableSortOrders] or null for default value
*/
@JvmSynthetic
@InternalParsersApi
@Deprecated(
"Use getList with filter instead",
replaceWith = ReplaceWith("getList(offset, filter)"),
)
abstract suspend fun getList(
offset: Int,
query: String?,
@@ -71,8 +104,15 @@ abstract class MangaParser @InternalParsersApi constructor(val source: MangaSour
* @param offset starting from 0 and used for pagination.
* @param query search query
*/
@Deprecated(
"Use getList with filter instead",
ReplaceWith(
"getList(offset, MangaListFilter.Search(query))",
"org.koitharu.kotatsu.parsers.model.MangaListFilter",
),
)
open suspend fun getList(offset: Int, query: String): List<Manga> {
return getList(offset, query, null, defaultSortOrder)
return getList(offset, MangaListFilter.Search(query))
}
/**
@@ -80,11 +120,29 @@ abstract class MangaParser @InternalParsersApi constructor(val source: MangaSour
*
* @param offset starting from 0 and used for pagination.
* Note than passed value may not be divisible by internal page size, so you should adjust it manually.
* @param tags genres for filtering, values from [getTags] and [Manga.tags]. May be null or empty
* @param sortOrder one of [sortOrders] or null for default value
* @param tags genres for filtering, values from [getAvailableTags] and [Manga.tags]. May be null or empty
* @param sortOrder one of [availableSortOrders] or null for default value
*/
@Deprecated(
"Use getList with filter instead",
ReplaceWith(
"getList(offset, MangaListFilter.Advanced(sortOrder, tags, null, emptySet()))",
"org.koitharu.kotatsu.parsers.model.MangaListFilter",
),
)
open suspend fun getList(offset: Int, tags: Set<MangaTag>?, sortOrder: SortOrder?): List<Manga> {
return getList(offset, null, tags, sortOrder ?: defaultSortOrder)
return getList(
offset,
MangaListFilter.Advanced(sortOrder ?: defaultSortOrder, tags.orEmpty(), null, emptySet()),
)
}
open suspend fun getList(offset: Int, filter: MangaListFilter?): List<Manga> {
return when (filter) {
is MangaListFilter.Advanced -> getList(offset, null, filter.tags, filter.sortOrder)
is MangaListFilter.Search -> getList(offset, filter.query, null, defaultSortOrder)
null -> getList(offset, null, null, defaultSortOrder)
}
}
/**
@@ -103,27 +161,29 @@ abstract class MangaParser @InternalParsersApi constructor(val source: MangaSour
/**
* Fetch direct link to the page image.
*/
open suspend fun getPageUrl(page: MangaPage): String = page.url.toAbsoluteUrl(getDomain())
open suspend fun getPageUrl(page: MangaPage): String = page.url.toAbsoluteUrl(domain)
/**
* Fetch available tags (genres) for source
*/
abstract suspend fun getTags(): Set<MangaTag>
abstract suspend fun getAvailableTags(): Set<MangaTag>
/**
* Returns direct link to the website favicon
* Fetch available locales for multilingual sources
*/
open suspend fun getAvailableLocales(): Set<Locale> = emptySet()
@Deprecated(
message = "Use parseFavicons() to get multiple favicons with different size",
replaceWith = ReplaceWith("parseFavicons()"),
message = "Use getAvailableTags instead",
replaceWith = ReplaceWith("getAvailableTags()"),
)
open fun getFaviconUrl() = "https://${getDomain()}/favicon.ico"
suspend fun getTags(): Set<MangaTag> = getAvailableTags()
/**
* Parse favicons from the main page of the source`s website
*/
open suspend fun getFavicons(): Favicons {
return FaviconParser(context, getDomain(), headers).parseFavicons()
return FaviconParser(webClient, domain).parseFavicons()
}
@CallSuper
@@ -131,70 +191,13 @@ abstract class MangaParser @InternalParsersApi constructor(val source: MangaSour
keys.add(configKeyDomain)
}
/* Utils */
fun getDomain(): String {
return config[configKeyDomain]
open suspend fun getRelatedManga(seed: Manga): List<Manga> {
return RelatedMangaFinder(listOf(this)).invoke(seed)
}
fun getDomain(subdomain: String): String {
val domain = getDomain()
return subdomain + "." + domain.removePrefix("www.")
protected fun getParser(source: MangaSource) = if (this.source == source) {
this
} else {
context.newParserInstance(source)
}
fun urlBuilder(): HttpUrl.Builder {
return HttpUrl.Builder()
.scheme("https")
.host(getDomain())
}
/**
* Create a unique id for [Manga]/[MangaChapter]/[MangaPage].
* @param url must be relative url, without a domain
* @see [Manga.id]
* @see [MangaChapter.id]
* @see [MangaPage.id]
*/
@InternalParsersApi
protected fun generateUid(url: String): Long {
var h = 1125899906842597L
source.name.forEach { c ->
h = 31 * h + c.code
}
url.forEach { c ->
h = 31 * h + c.code
}
return h
}
/**
* Create a unique id for [Manga]/[MangaChapter]/[MangaPage].
* @param id an internal identifier
* @see [Manga.id]
* @see [MangaChapter.id]
* @see [MangaPage.id]
*/
@InternalParsersApi
protected fun generateUid(id: Long): Long {
var h = 1125899906842597L
source.name.forEach { c ->
h = 31 * h + c.code
}
h = 31 * h + id
return h
}
@InternalParsersApi
protected fun Element.parseFailed(message: String? = null): Nothing {
throw ParseException(message, ownerDocument()?.location() ?: baseUri(), null)
}
@InternalParsersApi
protected fun Set<MangaTag>?.oneOrThrowIfMany(): MangaTag? {
return when {
isNullOrEmpty() -> null
size == 1 -> first()
else -> throw IllegalArgumentException("Multiple genres are not supported by this source")
}
}
}
}

View File

@@ -1,5 +1,7 @@
package org.koitharu.kotatsu.parsers
import org.koitharu.kotatsu.parsers.model.ContentType
/**
* Annotate each [MangaParser] implementation with this annotation, used by codegen
*/
@@ -11,10 +13,15 @@ annotation class MangaSourceParser(
val name: String,
/**
* User-friendly title of manga source. In most case equals the website name.
* Avoid extra whitespaces between the words if it is not required.
*/
val title: String,
/**
* Language code (for example "en" or "ru") or blank if parser provide manga on different languages.
*/
val locale: String = "",
)
/**
* Type of content provided by parser. See [ContentType] for more info
*/
val type: ContentType = ContentType.MANGA,
)

View File

@@ -1,5 +1,6 @@
package org.koitharu.kotatsu.parsers
import androidx.annotation.RestrictTo
import org.koitharu.kotatsu.parsers.model.Manga
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.model.MangaTag
@@ -8,12 +9,16 @@ import org.koitharu.kotatsu.parsers.util.Paginator
@InternalParsersApi
abstract class PagedMangaParser(
context: MangaLoaderContext,
source: MangaSource,
pageSize: Int,
@RestrictTo(RestrictTo.Scope.TESTS) @JvmField internal val pageSize: Int,
searchPageSize: Int = pageSize,
) : MangaParser(source) {
) : MangaParser(context, source) {
@JvmField
protected val paginator = Paginator(pageSize)
@JvmField
protected val searchPaginator = Paginator(searchPageSize)
override suspend fun getList(offset: Int, query: String): List<Manga> {
@@ -47,4 +52,4 @@ abstract class PagedMangaParser(
paginator.onListReceived(offset, page, list.size)
return list
}
}
}

View File

@@ -1,13 +1,28 @@
package org.koitharu.kotatsu.parsers.config
sealed class ConfigKey<T>(
val key: String,
@JvmField val key: String,
) {
abstract val defaultValue: T
class Domain(
@JvmField @JvmSuppressWildcards vararg val presetValues: String,
) : ConfigKey<String>("domain") {
init {
require(presetValues.isNotEmpty()) { "You must provide at least one domain" }
}
override val defaultValue: String
get() = presetValues.first()
}
class ShowSuspiciousContent(
override val defaultValue: Boolean,
) : ConfigKey<Boolean>("show_suspicious")
class UserAgent(
override val defaultValue: String,
val presetValues: Array<String>?,
) : ConfigKey<String>("domain")
}
) : ConfigKey<String>("user_agent")
}

View File

@@ -6,6 +6,7 @@ import org.koitharu.kotatsu.parsers.model.MangaSource
/**
* Authorization is required for access to the requested content
*/
class AuthRequiredException @InternalParsersApi constructor(
class AuthRequiredException @InternalParsersApi @JvmOverloads constructor(
val source: MangaSource,
) : RuntimeException("Authorization required")
cause: Throwable? = null,
) : RuntimeException("Authorization required", cause)

View File

@@ -1,7 +0,0 @@
package org.koitharu.kotatsu.parsers.exception
import okio.IOException
class CloudFlareProtectedException(
val url: String,
) : IOException("Protected by CloudFlare: $url")

View File

@@ -0,0 +1,24 @@
package org.koitharu.kotatsu.parsers.model
enum class ContentType {
/**
* Standard manga, manhua, webtoons, etc
*/
MANGA,
/**
* Use this if the source provides mostly nsfw content.
*/
HENTAI,
/**
* Western comics
*/
COMICS,
/**
* Use this type if no other suits your needs. For example, for an indie manga
*/
OTHER,
}

View File

@@ -3,11 +3,12 @@ package org.koitharu.kotatsu.parsers.model
import okhttp3.HttpUrl.Companion.toHttpUrl
class Favicon internal constructor(
val url: String,
val size: Int,
internal val rel: String?,
@JvmField val url: String,
@JvmField val size: Int,
@JvmField internal val rel: String?,
) : Comparable<Favicon> {
@JvmField
val type: String = url.toHttpUrl().pathSegments.last()
.substringAfterLast('.', "").lowercase()
@@ -48,4 +49,4 @@ class Favicon internal constructor(
"mask-icon" -> -1
else -> 0
}
}
}

View File

@@ -2,7 +2,7 @@ package org.koitharu.kotatsu.parsers.model
class Favicons internal constructor(
favicons: Collection<Favicon>,
val referer: String,
@JvmField val referer: String,
) : Collection<Favicon> {
private val icons = favicons.sortedDescending()
@@ -18,6 +18,11 @@ class Favicons internal constructor(
override fun iterator(): Iterator<Favicon> = icons.iterator()
operator fun minus(victim: Favicon): Favicons = Favicons(
favicons = icons.filterNot { it == victim },
referer = referer,
)
/**
* Finds a favicon whose size in pixels is greater than or equal to the specified size.
* If such icon is not available returns the largest icon
@@ -42,4 +47,4 @@ class Favicons internal constructor(
}
return result
}
}
}

View File

@@ -6,67 +6,67 @@ class Manga(
/**
* Unique identifier for manga
*/
val id: Long,
@JvmField val id: Long,
/**
* Manga title, human-readable
*/
val title: String,
@JvmField val title: String,
/**
* Alternative title (for example on other language), may be null
*/
val altTitle: String?,
@JvmField val altTitle: String?,
/**
* Relative url to manga (**without** a domain) or any other uri.
* Used principally in parsers
*/
val url: String,
@JvmField val url: String,
/**
* Absolute url to manga, must be ready to open in browser
*/
val publicUrl: String,
@JvmField val publicUrl: String,
/**
* Normalized manga rating, must be in range of 0..1 or [RATING_UNKNOWN] if rating s unknown
* @see hasRating
*/
val rating: Float,
@JvmField val rating: Float,
/**
* Indicates that manga may contain sensitive information (18+, NSFW)
*/
val isNsfw: Boolean,
@JvmField val isNsfw: Boolean,
/**
* Absolute link to the cover
* @see largeCoverUrl
*/
val coverUrl: String,
@JvmField val coverUrl: String,
/**
* Tags (genres) of the manga
*/
val tags: Set<MangaTag>,
@JvmField val tags: Set<MangaTag>,
/**
* Manga status (ongoing, finished) or null if unknown
*/
val state: MangaState?,
@JvmField val state: MangaState?,
/**
* Author of the manga, may be null
*/
val author: String?,
@JvmField val author: String?,
/**
* Large cover url (absolute), null if is no large cover
* @see coverUrl
*/
val largeCoverUrl: String? = null,
@JvmField val largeCoverUrl: String? = null,
/**
* Manga description, may be html or null
*/
val description: String? = null,
@JvmField val description: String? = null,
/**
* List of chapters
*/
val chapters: List<MangaChapter>? = null,
@JvmField val chapters: List<MangaChapter>? = null,
/**
* Manga source
*/
val source: MangaSource,
@JvmField val source: MangaSource,
) {
/**
@@ -155,4 +155,8 @@ class Manga(
result = 31 * result + source.hashCode()
return result
}
}
override fun toString(): String {
return "Manga($id - \"$title\" [$url] - $source)"
}
}

View File

@@ -4,34 +4,34 @@ class MangaChapter(
/**
* An unique id of chapter
*/
val id: Long,
@JvmField val id: Long,
/**
* User-readable name of chapter
*/
val name: String,
@JvmField val name: String,
/**
* Chapter number starting from 1
*/
val number: Int,
@JvmField val number: Int,
/**
* Relative url to chapter (**without** a domain) or any other uri.
* Used principally in parsers
*/
val url: String,
@JvmField val url: String,
/**
* User-readable name of scanlator (releaser) or null if unknown
*/
val scanlator: String?,
@JvmField val scanlator: String?,
/**
* Chapter upload date in milliseconds
*/
val uploadDate: Long,
@JvmField val uploadDate: Long,
/**
* User-readable name of branch.
* A branch is a group of chapters that overlap (e.g. different languages)
*/
val branch: String?,
val source: MangaSource,
@JvmField val branch: String?,
@JvmField val source: MangaSource,
) : Comparable<MangaChapter> {
override fun compareTo(other: MangaChapter): Int {
@@ -67,4 +67,8 @@ class MangaChapter(
result = 31 * result + source.hashCode()
return result
}
}
override fun toString(): String {
return "MangaChapter($id - #$number [$url] - $source)"
}
}

View File

@@ -0,0 +1,29 @@
package org.koitharu.kotatsu.parsers.model
import java.util.*
sealed interface MangaListFilter {
fun isEmpty(): Boolean
val sortOrder: SortOrder?
data class Search(
@JvmField val query: String,
) : MangaListFilter {
override val sortOrder: SortOrder? = null
override fun isEmpty() = query.isBlank()
}
data class Advanced(
override val sortOrder: SortOrder,
@JvmField val tags: Set<MangaTag>,
@JvmField val locale: Locale?,
@JvmField val states: Set<MangaState>,
) : MangaListFilter {
override fun isEmpty(): Boolean = tags.isEmpty() && locale == null && states.isEmpty()
}
}

View File

@@ -6,24 +6,19 @@ class MangaPage(
/**
* Unique identifier for manga
*/
val id: Long,
@JvmField val id: Long,
/**
* Relative url to page (**without** a domain) or any other uri.
* Used principally in parsers.
* May contain link to image or html page.
* @see MangaParser.getPageUrl
*/
val url: String,
/**
* Absolute link to the chapter or website home page.
* Used in Referer header
*/
val referer: String,
@JvmField val url: String,
/**
* Absolute url of the small page image if exists, null otherwise
*/
val preview: String?,
val source: MangaSource,
@JvmField val preview: String?,
@JvmField val source: MangaSource,
) {
override fun equals(other: Any?): Boolean {
@@ -34,19 +29,19 @@ class MangaPage(
if (id != other.id) return false
if (url != other.url) return false
if (referer != other.referer) return false
if (preview != other.preview) return false
if (source != other.source) return false
return true
return source == other.source
}
override fun hashCode(): Int {
var result = id.hashCode()
result = 31 * result + url.hashCode()
result = 31 * result + referer.hashCode()
result = 31 * result + (preview?.hashCode() ?: 0)
result = 31 * result + source.hashCode()
return result
}
}
override fun toString(): String {
return "MangaPage($id [$url] - $source)"
}
}

View File

@@ -1,5 +1,5 @@
package org.koitharu.kotatsu.parsers.model
enum class MangaState {
ONGOING, FINISHED
}
ONGOING, FINISHED, ABANDONED, PAUSED
}

View File

@@ -6,13 +6,13 @@ class MangaTag(
/**
* User-readable tag title, should be in Title case
*/
val title: String,
@JvmField val title: String,
/**
* Identifier of a tag, must be unique among the source.
* @see MangaParser.getList
*/
val key: String,
val source: MangaSource,
@JvmField val key: String,
@JvmField val source: MangaSource,
) {
override fun equals(other: Any?): Boolean {
@@ -34,4 +34,8 @@ class MangaTag(
result = 31 * result + source.hashCode()
return result
}
}
override fun toString(): String {
return "MangaTag($key \"$title\" - $source)"
}
}

View File

@@ -0,0 +1,133 @@
package org.koitharu.kotatsu.parsers.network
import okhttp3.*
import okhttp3.MediaType.Companion.toMediaType
import okhttp3.RequestBody.Companion.toRequestBody
import org.json.JSONObject
import org.jsoup.HttpStatusException
import org.koitharu.kotatsu.parsers.exception.AuthRequiredException
import org.koitharu.kotatsu.parsers.exception.GraphQLException
import org.koitharu.kotatsu.parsers.exception.NotFoundException
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.util.await
import org.koitharu.kotatsu.parsers.util.parseJson
import java.net.HttpURLConnection
class OkHttpWebClient(
private val httpClient: OkHttpClient,
private val mangaSource: MangaSource,
) : WebClient {
override suspend fun httpGet(url: HttpUrl, extraHeaders: Headers?): Response {
val request = Request.Builder()
.get()
.url(url)
.addTags()
.addExtraHeaders(extraHeaders)
return httpClient.newCall(request.build()).await().ensureSuccess()
}
override suspend fun httpHead(url: HttpUrl): Response {
val request = Request.Builder()
.head()
.url(url)
.addTags()
return httpClient.newCall(request.build()).await().ensureSuccess()
}
override suspend fun httpPost(url: HttpUrl, form: Map<String, String>, extraHeaders: Headers?): Response {
val body = FormBody.Builder()
form.forEach { (k, v) ->
body.addEncoded(k, v)
}
val request = Request.Builder()
.post(body.build())
.url(url)
.addTags()
.addExtraHeaders(extraHeaders)
return httpClient.newCall(request.build()).await().ensureSuccess()
}
override suspend fun httpPost(url: HttpUrl, payload: String, extraHeaders: Headers?): Response {
val body = FormBody.Builder()
payload.split('&').forEach {
val pos = it.indexOf('=')
if (pos != -1) {
val k = it.substring(0, pos)
val v = it.substring(pos + 1)
body.addEncoded(k, v)
}
}
val request = Request.Builder()
.post(body.build())
.url(url)
.addTags()
.addExtraHeaders(extraHeaders)
return httpClient.newCall(request.build()).await().ensureSuccess()
}
override suspend fun httpPost(url: HttpUrl, body: JSONObject, extraHeaders: Headers?): Response {
val mediaType = "application/json; charset=utf-8".toMediaType()
val requestBody = body.toString().toRequestBody(mediaType)
val request = Request.Builder()
.post(requestBody)
.url(url)
.addTags()
.addExtraHeaders(extraHeaders)
return httpClient.newCall(request.build()).await().ensureSuccess()
}
override suspend fun graphQLQuery(endpoint: String, query: String): JSONObject {
val body = JSONObject()
body.put("operationName", null as Any?)
body.put("variables", JSONObject())
body.put("query", "{$query}")
val mediaType = "application/json; charset=utf-8".toMediaType()
val requestBody = body.toString().toRequestBody(mediaType)
val request = Request.Builder()
.post(requestBody)
.url(endpoint)
.addTags()
val json = httpClient.newCall(request.build()).await().parseJson()
json.optJSONArray("errors")?.let {
if (it.length() != 0) {
throw GraphQLException(it)
}
}
return json
}
private fun Request.Builder.addTags(): Request.Builder {
tag(MangaSource::class.java, mangaSource)
return this
}
private fun Request.Builder.addExtraHeaders(headers: Headers?): Request.Builder {
if (headers != null) {
headers(headers)
}
return this
}
private fun Response.ensureSuccess(): Response {
val exception: Exception? = when (code) { // Catch some error codes, not all
HttpURLConnection.HTTP_NOT_FOUND -> NotFoundException(message, request.url.toString())
HttpURLConnection.HTTP_UNAUTHORIZED -> request.tag(MangaSource::class.java)?.let {
AuthRequiredException(it)
} ?: HttpStatusException(message, code, request.url.toString())
in 400..599 -> HttpStatusException(message, code, request.url.toString())
else -> null
}
if (exception != null) {
runCatching {
close()
}.onFailure {
exception.addSuppressed(it)
}
throw exception
}
return this
}
}

View File

@@ -0,0 +1,14 @@
package org.koitharu.kotatsu.parsers.network
object UserAgents {
const val CHROME_MOBILE =
"Mozilla/5.0 (Linux; Android 13) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.5735.196 Mobile Safari/537.36"
const val CHROME_DESKTOP =
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36"
const val FIREFOX_DESKTOP = "Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/116.0"
const val KOTATSU = "Kotatsu/5.3 (Android 13;;; en)"
}

View File

@@ -0,0 +1,116 @@
package org.koitharu.kotatsu.parsers.network
import okhttp3.Headers
import okhttp3.HttpUrl
import okhttp3.HttpUrl.Companion.toHttpUrl
import okhttp3.Response
import org.json.JSONObject
interface WebClient {
/**
* Do a GET http request to specific url
* @param url
*/
suspend fun httpGet(url: String): Response = httpGet(url.toHttpUrl())
suspend fun httpGet(url: String, extraHeaders: Headers?): Response = httpGet(url.toHttpUrl(), extraHeaders)
/**
* Do a GET http request to specific url
* @param url
*/
suspend fun httpGet(url: HttpUrl): Response = httpGet(url, null)
/**
* Do a GET http request to specific url
* @param url
* @param extraHeaders additional HTTP headers for request
*/
suspend fun httpGet(url: HttpUrl, extraHeaders: Headers?): Response
/**
* Do a HEAD http request to specific url
* @param url
*/
suspend fun httpHead(url: String): Response = httpHead(url.toHttpUrl())
/**
* Do a HEAD http request to specific url
* @param url
*/
suspend fun httpHead(url: HttpUrl): Response
/**
* Do a POST http request to specific url with `multipart/form-data` payload
* @param url
* @param form payload as key=>value map
*/
suspend fun httpPost(url: String, form: Map<String, String>): Response = httpPost(url.toHttpUrl(), form, null)
/**
* Do a POST http request to specific url with `multipart/form-data` payload
* @param url
* @param form payload as key=>value map
*/
suspend fun httpPost(url: HttpUrl, form: Map<String, String>): Response = httpPost(url, form, null)
/**
* Do a POST http request to specific url with `multipart/form-data` payload
* @param url
* @param form payload as key=>value map
* @param extraHeaders additional HTTP headers for request
*/
suspend fun httpPost(url: HttpUrl, form: Map<String, String>, extraHeaders: Headers?): Response
/**
* Do a POST http request to specific url with `multipart/form-data` payload
* @param url
* @param payload payload as `key=value` string with `&` separator
*/
suspend fun httpPost(url: String, payload: String): Response = httpPost(url.toHttpUrl(), payload, null)
/**
* Do a POST http request to specific url with `multipart/form-data` payload
* @param url
* @param payload payload as `key=value` string with `&` separator
*/
suspend fun httpPost(url: HttpUrl, payload: String): Response = httpPost(url, payload, null)
/**
* Do a POST http request to specific url with `multipart/form-data` payload
* @param url
* @param payload payload as `key=value` string with `&` separator
* @param extraHeaders additional HTTP headers for request
*/
suspend fun httpPost(url: HttpUrl, payload: String, extraHeaders: Headers?): Response
/**
* Do a POST http request to specific url with json payload
* @param url
* @param body
*/
suspend fun httpPost(url: String, body: JSONObject): Response = httpPost(url.toHttpUrl(), body, null)
/**
* Do a POST http request to specific url with json payload
* @param url
* @param body
*/
suspend fun httpPost(url: HttpUrl, body: JSONObject): Response = httpPost(url, body, null)
/**
* Do a POST http request to specific url with json payload
* @param url
* @param body
* @param extraHeaders additional HTTP headers for request
*/
suspend fun httpPost(url: HttpUrl, body: JSONObject, extraHeaders: Headers?): Response
/**
* Do a GraphQL request to specific url
* @param endpoint an url
* @param query GraphQL request payload
*/
suspend fun graphQLQuery(endpoint: String, query: String): JSONObject
}

View File

@@ -1,189 +0,0 @@
package org.koitharu.kotatsu.parsers.site
import org.json.JSONObject
import org.jsoup.nodes.Element
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import org.koitharu.kotatsu.parsers.util.json.associateByKey
import org.koitharu.kotatsu.parsers.util.json.stringIterator
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("KOMIKTAP", "KomikTap", "id")
class KomikTapParser(
override val context: MangaLoaderContext,
) : PagedMangaParser(MangaSource.KOMIKTAP, pageSize = 25, searchPageSize = 10) {
override val configKeyDomain = ConfigKey.Domain("194.233.66.232", arrayOf("194.233.66.232", "komiktap.in"))
override val sortOrders = EnumSet.of(
SortOrder.ALPHABETICAL,
SortOrder.POPULARITY,
SortOrder.UPDATED,
SortOrder.NEWEST,
)
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val url = urlBuilder()
if (query.isNullOrEmpty()) {
url.addPathSegment("manga")
.addQueryParameter("page", page.toString())
.addQueryParameter("status", "")
.addQueryParameter("type", "")
.addQueryParameter("order", sortOrder.asQueryParameter())
tags?.forEach {
url.addQueryParameter("genre[]", it.key)
}
} else {
url.addPathSegment("page")
.addPathSegment(page.toString())
.addQueryParameter("s", query)
}
val root = context.httpGet(url.build()).parseHtml().body()
.requireElementById("content")
.selectFirstOrThrow(".listupd")
return root.select("div.bs").map { div ->
val a = div.selectFirstOrThrow("a")
val img = div.selectFirstOrThrow("img")
val href = a.attrAsRelativeUrl("href")
val bigor = div.selectFirstOrThrow(".bigor")
Manga(
id = generateUid(href),
title = bigor.selectFirstOrThrow(".tt").text(),
altTitle = null,
url = href,
publicUrl = a.attrAsAbsoluteUrl("href"),
rating = bigor.selectFirst(".rating .numscore")
?.text()?.toFloatOrNull()?.div(10f)
.assertNotNull("rating") ?: RATING_UNKNOWN,
isNsfw = true,
coverUrl = img.attrAsAbsoluteUrl("src"),
tags = emptySet(),
state = when (div.selectFirst("span.status")?.text()) {
"Completed" -> MangaState.FINISHED
else -> null
},
author = null,
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga {
val root = context.httpGet(manga.url.toAbsoluteUrl(getDomain())).parseHtml().body()
.requireElementById("content")
.selectFirstOrThrow("article")
val table = root.selectFirstOrThrow(".infotable")
val dateFormat = SimpleDateFormat("MMM d, yyyy", checkNotNull(sourceLocale))
val chapters = root.requireElementById("chapterlist")
.select("li")
.mapChapters { index, li ->
val a = li.selectFirstOrThrow("a")
val href = a.attrAsRelativeUrl("href")
MangaChapter(
id = generateUid(href),
name = a.selectFirstOrThrow(".chapternum").text(),
number = index + 1,
url = href,
scanlator = null,
uploadDate = dateFormat.tryParse(a.selectFirst(".chapterdate")?.text()),
branch = null,
source = source,
)
}
return manga.copy(
largeCoverUrl = root.selectFirst("div.thumb")
?.selectFirst("img")
?.attrAsAbsoluteUrlOrNull("src")
.assertNotNull("largeCoverUrl"),
author = table.tableValue("Author").assertNotNull("author")?.takeUnless { it == "N/A" },
state = when (table.tableValue("Status").assertNotNull("status")) {
"Completed" -> MangaState.FINISHED
"Ongoing" -> MangaState.ONGOING
else -> null
},
tags = root.selectFirstOrThrow(".seriestugenre")
.select("a")
.mapToSet { a ->
MangaTag(
title = a.text().toTitleCase(Locale.ENGLISH),
key = a.attr("href").removeSuffix('/').substringAfterLast('/'),
source = manga.source,
)
},
description = root.selectFirstOrThrow("[itemprop=\"description\"]").html(),
chapters = chapters,
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(getDomain())
val doc = context.httpGet(fullUrl).parseHtml()
val scripts = doc.select("script")
for (script in scripts) {
val content = script.html()
val pos = content.indexOf("ts_reader.run")
if (pos < 0) {
continue
}
val json = JSONObject(content.substringBetween("(", ")"))
val sources = json.getJSONArray("sources").associateByKey("source")
val images = (sources[json.optString("defaultSource")] ?: sources.values.first()).getJSONArray("images")
val result = ArrayList<MangaPage>(images.length())
images.stringIterator().forEach {
result += MangaPage(
id = generateUid(it),
url = it,
referer = fullUrl,
preview = null,
source = source,
)
}
return result
}
doc.parseFailed("Script with pages not found")
}
override suspend fun getTags(): Set<MangaTag> {
val root = context.httpGet("https://${getDomain()}/manga/").parseHtml()
.selectFirstOrThrow("form.filters")
.selectFirstOrThrow("ul.genrez")
return root.select("li").mapNotNullToSet { li ->
val input = li.selectFirstOrThrow("input")
if (input.attr("name") != "genre[]") {
return@mapNotNullToSet null
}
MangaTag(
title = li.selectFirstOrThrow("label").text().toTitleCase(sourceLocale ?: Locale.ENGLISH),
key = input.attrOrNull("value") ?: return@mapNotNullToSet null,
source = source,
)
}
}
override fun getFaviconUrl(): String {
return "https://${getDomain()}/wp-content/uploads/2020/09/cropped-LOGOa-180x180.png"
}
private fun SortOrder.asQueryParameter() = when (this) {
SortOrder.UPDATED -> "update"
SortOrder.POPULARITY -> "popular"
SortOrder.NEWEST -> "latest"
SortOrder.ALPHABETICAL -> "title"
else -> ""
}
private fun Element.tableValue(key: String): String? {
return getElementsMatchingOwnText(key).singleOrNull()?.parent()?.selectLast("td")?.text()
}
}

View File

@@ -1,186 +0,0 @@
package org.koitharu.kotatsu.parsers.site
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaParser
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("MANGAOWL", "MangaOwl", "en")
internal class MangaOwlParser(override val context: MangaLoaderContext) : MangaParser(MangaSource.MANGAOWL) {
override val configKeyDomain = ConfigKey.Domain("mangaowls.com", null)
override val sortOrders: Set<SortOrder> = EnumSet.of(
SortOrder.POPULARITY,
SortOrder.NEWEST,
SortOrder.UPDATED,
)
private val regexNsfw = Regex("(yaoi)|(yuri)|(smut)|(mature)|(adult)", RegexOption.IGNORE_CASE)
override suspend fun getList(
offset: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val page = (offset / 36f).toIntUp().inc()
val link = buildString {
append("https://")
append(getDomain())
when {
!query.isNullOrEmpty() -> {
append("/search/$page?search=")
append(query.urlEncoded())
}
!tags.isNullOrEmpty() -> {
for (tag in tags) {
append(tag.key)
}
append("/$page?type=${getAlternativeSortKey(sortOrder)}")
}
else -> {
append("/${getSortKey(sortOrder)}/$page")
}
}
}
val doc = context.httpGet(link).parseHtml()
val slides = doc.body().selectOrThrow("ul.slides")
val items = slides.select("div.col-md-2")
return items.mapNotNull { item ->
val href = item.selectFirst("h6 a")?.attrAsRelativeUrlOrNull("href") ?: return@mapNotNull null
Manga(
id = generateUid(href),
title = item.selectFirst("h6 a")?.text() ?: return@mapNotNull null,
coverUrl = item.select("div.img-responsive").attr("abs:data-background-image"),
altTitle = null,
author = null,
rating = runCatching {
item.selectFirst("div.block-stars")
?.text()
?.toFloatOrNull()
?.div(10f)
}.getOrNull() ?: RATING_UNKNOWN,
url = href,
isNsfw = false,
tags = emptySet(),
state = null,
publicUrl = href.toAbsoluteUrl(getDomain()),
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga {
val doc = context.httpGet(manga.publicUrl).parseHtml()
val info = doc.body().selectFirstOrThrow("div.single_detail")
val table = doc.body().selectFirstOrThrow("div.single-grid-right")
val dateFormat = SimpleDateFormat("MM/dd/yyyy", Locale.US)
val trRegex = "window\\['tr'] = '([^']*)';".toRegex(RegexOption.IGNORE_CASE)
val trElement = doc.getElementsByTag("script").find { trRegex.find(it.data()) != null }
?: doc.parseFailed("Oops, tr not found")
val tr = trRegex.find(trElement.data())!!.groups[1]!!.value
val s = context.encodeBase64(getDomain().toByteArray())
var isNsfw = manga.isNsfw
val parsedTags = info.select("div.col-xs-12.col-md-8.single-right-grid-right > p > a[href*=genres]")
.mapNotNullToSet {
val a = it.selectFirst("a") ?: return@mapNotNullToSet null
val name = a.text()
if (!isNsfw && isNsfwGenre(name)) {
isNsfw = true
}
MangaTag(
title = name.toTitleCase(),
key = a.attr("href"),
source = source,
)
}
return manga.copy(
description = info.selectFirst(".description")?.html(),
largeCoverUrl = info.select("img").first()?.let { img ->
if (img.hasAttr("data-src")) img.attr("abs:data-src") else img.attr("abs:src")
},
isNsfw = isNsfw,
author = info.selectFirst("p.fexi_header_para a.author_link")?.text(),
state = parseStatus(info.select("p.fexi_header_para:contains(status)").first()?.ownText()),
tags = manga.tags + parsedTags,
chapters = table.select("div.table.table-chapter-list").select("li.list-group-item.chapter_list")
.asReversed().mapChapters { i, li ->
val a = li.select("a")
val href = a.attr("data-href").ifEmpty {
li.parseFailed("Link is missing")
}
MangaChapter(
id = generateUid(href),
name = a.select("label").text(),
number = i + 1,
url = "$href?tr=$tr&s=$s",
scanlator = null,
branch = null,
uploadDate = dateFormat.tryParse(li.selectFirst("small:last-of-type")?.text()),
source = MangaSource.MANGAOWL,
)
},
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(getDomain())
val doc = context.httpGet(fullUrl).parseHtml()
val root = doc.body().selectOrThrow("div.item img.owl-lazy")
return root.map { div ->
val url = div?.attrAsRelativeUrlOrNull("data-src") ?: doc.parseFailed("Page image not found")
MangaPage(
id = generateUid(url),
url = url,
preview = null,
referer = url,
source = MangaSource.MANGAOWL,
)
}
}
private fun parseStatus(status: String?) = when {
status == null -> null
status.contains("Ongoing") -> MangaState.ONGOING
status.contains("Completed") -> MangaState.FINISHED
else -> null
}
override suspend fun getTags(): Set<MangaTag> {
val doc = context.httpGet("https://${getDomain()}/").parseHtml()
val root = doc.body().select("ul.dropdown-menu.multi-column.columns-3").select("li")
return root.mapToSet { p ->
val a = p.selectFirstOrThrow("a")
MangaTag(
title = a.text().toTitleCase(),
key = a.attr("href"),
source = source,
)
}
}
private fun getSortKey(sortOrder: SortOrder) =
when (sortOrder) {
SortOrder.POPULARITY -> "popular"
SortOrder.NEWEST -> "new_release"
SortOrder.UPDATED -> "lastest"
else -> "lastest"
}
private fun getAlternativeSortKey(sortOrder: SortOrder) =
when (sortOrder) {
SortOrder.POPULARITY -> "0"
SortOrder.NEWEST -> "2"
SortOrder.UPDATED -> "3"
else -> "3"
}
private fun isNsfwGenre(name: String): Boolean = regexNsfw.containsMatchIn(name)
}

View File

@@ -1,188 +0,0 @@
package org.koitharu.kotatsu.parsers.site
import androidx.collection.ArraySet
import kotlinx.coroutines.async
import kotlinx.coroutines.awaitAll
import kotlinx.coroutines.coroutineScope
import org.jsoup.nodes.Element
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("NHENTAI", "N-Hentai")
class NHentaiParser(override val context: MangaLoaderContext) : PagedMangaParser(MangaSource.NHENTAI, pageSize = 25) {
override val configKeyDomain: ConfigKey.Domain
get() = ConfigKey.Domain("nhentai.net", null)
override val sortOrders: Set<SortOrder>
get() = EnumSet.of(SortOrder.NEWEST, SortOrder.POPULARITY)
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
if (query.isNullOrEmpty() && tags != null && tags.size > 1) {
return getListPage(page, buildQuery(tags), emptySet(), sortOrder)
}
val domain = getDomain()
val url = buildString {
append("https://")
append(domain)
if (!query.isNullOrEmpty()) {
append("/search/?q=")
append(query.urlEncoded())
append("&page=")
append(page)
if (sortOrder == SortOrder.POPULARITY) {
append("&sort=popular")
}
} else {
append('/')
if (!tags.isNullOrEmpty()) {
val tag = tags.single()
append("tag/")
append(tag.key)
append('/')
if (sortOrder == SortOrder.POPULARITY) {
append("popular")
}
append("?page=")
append(page)
} else {
if (sortOrder == SortOrder.POPULARITY) {
append("?sort=popular&page=")
} else {
append("?page=")
}
append(page)
}
}
}
val root = context.httpGet(url).parseHtml().body().requireElementById("content")
.selectLastOrThrow("div.index-container")
val regexBrackets = Regex("\\[[^]]+]|\\([^)]+\\)")
val regexSpaces = Regex("\\s+")
return root.select(".gallery").map { div ->
val a = div.selectFirstOrThrow("a.cover")
val href = a.attrAsRelativeUrl("href")
val img = div.selectFirstOrThrow("img")
val title = div.selectFirstOrThrow(".caption").text()
Manga(
id = generateUid(href),
title = title.replace(regexBrackets, "")
.replace(regexSpaces, " ")
.trim(),
altTitle = null,
url = href,
publicUrl = href.toAbsoluteUrl(domain),
rating = RATING_UNKNOWN,
isNsfw = true,
coverUrl = img.attrAsAbsoluteUrlOrNull("data-src")
?: img.attrAsAbsoluteUrl("src"),
tags = setOf(),
state = null,
author = null,
largeCoverUrl = null,
description = null,
chapters = listOf(),
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga {
val root = context.httpGet(
url = manga.url.toAbsoluteUrl(getDomain()),
).parseHtml().body().requireElementById("bigcontainer")
val img = root.requireElementById("cover").selectFirstOrThrow("img")
val tagContainers = root.requireElementById("tags").select(".tag-container")
val dateFormat = SimpleDateFormat(
"yyyy-MM-dd'T'HH:mm:ss.SSSSSS'+00:00'",
Locale.ROOT,
)
return manga.copy(
tags = tagContainers.find { x -> x.ownText() == "Tags:" }?.parseTags() ?: manga.tags,
author = tagContainers.find { x -> x.ownText() == "Artists:" }
?.selectFirst("span.name")?.text()?.toCamelCase(),
largeCoverUrl = img.attrAsAbsoluteUrlOrNull("data-src")
?: img.attrAsAbsoluteUrl("src"),
description = null,
chapters = listOf(
MangaChapter(
id = manga.id,
name = manga.title,
number = 1,
url = manga.url,
scanlator = null,
uploadDate = dateFormat.tryParse(
tagContainers.find { x -> x.ownText() == "Uploaded:" }
?.selectFirst("time")
?.attr("datetime"),
),
branch = null,
source = source,
),
),
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val url = chapter.url.toAbsoluteUrl(getDomain())
val root = context.httpGet(url).parseHtml().requireElementById("thumbnail-container")
return root.select(".thumb-container").map { div ->
val a = div.selectFirstOrThrow("a")
val img = div.selectFirstOrThrow("img")
val href = a.attrAsRelativeUrl("href")
MangaPage(
id = generateUid(href),
url = href,
referer = url,
preview = img.attrAsAbsoluteUrlOrNull("data-src")
?: img.attrAsAbsoluteUrl("src"),
source = source,
)
}
}
override suspend fun getPageUrl(page: MangaPage): String {
val root = context.httpGet(page.url.toAbsoluteUrl(getDomain())).parseHtml().body()
.requireElementById("image-container")
return root.selectFirstOrThrow("img").attrAsAbsoluteUrl("src")
}
override suspend fun getTags(): Set<MangaTag> {
return coroutineScope {
// parse first 3 pages of tags
(1..3).map { page ->
async { getTags(page) }
}
}.awaitAll().flattenTo(ArraySet(360))
}
private suspend fun getTags(page: Int): Set<MangaTag> {
val root = context.httpGet("https://${getDomain()}/tags/popular?page=$page").parseHtml().body()
.getElementById("tag-container")
return root?.parseTags().orEmpty()
}
private fun Element.parseTags() = select("a.tag").mapToSet { a ->
val href = a.attr("href").removeSuffix('/')
MangaTag(
title = a.selectFirstOrThrow(".name").text().toTitleCase(),
key = href.substringAfterLast('/'),
source = source,
)
}
private fun buildQuery(tags: Collection<MangaTag>) = tags.joinToString(separator = " ") { tag ->
"tag:\"${tag.key}\""
}
}

View File

@@ -1,228 +0,0 @@
package org.koitharu.kotatsu.parsers.site
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaParser
import org.koitharu.kotatsu.parsers.MangaParserAuthProvider
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.exception.AuthRequiredException
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.text.SimpleDateFormat
import java.util.*
private const val MAX_THUMB_INDEX = 19
@MangaSourceParser("NUDEMOON", "Nude-Moon", "ru")
internal class NudeMoonParser(
override val context: MangaLoaderContext,
) : MangaParser(MangaSource.NUDEMOON), MangaParserAuthProvider {
override val configKeyDomain = ConfigKey.Domain("nude-moon.net", null)
override val authUrl: String
get() = "https://${getDomain()}/index.php"
override val isAuthorized: Boolean
get() {
return context.cookieJar.getCookies(getDomain()).any {
it.name == "fusion_user"
}
}
override val sortOrders: Set<SortOrder> = EnumSet.of(
SortOrder.NEWEST,
SortOrder.POPULARITY,
SortOrder.RATING,
)
init {
context.cookieJar.insertCookies(
getDomain(),
"NMfYa=1;",
"nm_mobile=0;",
)
}
override suspend fun getList(
offset: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val domain = getDomain()
val url = when {
!query.isNullOrEmpty() -> "https://$domain/search?stext=${query.urlEncoded()}&rowstart=$offset"
!tags.isNullOrEmpty() -> tags.joinToString(
separator = "_",
prefix = "https://$domain/tags/",
postfix = "&rowstart=$offset",
transform = { it.key.urlEncoded() },
)
else -> "https://$domain/all_manga?${getSortKey(sortOrder)}&rowstart=$offset"
}
val doc = context.httpGet(url).parseHtml()
val root = doc.body().run {
selectFirst("td.main-bg") ?: selectFirst("td.main-body")
} ?: doc.parseFailed("Cannot find root")
return root.select("table.news_pic2").mapNotNull { row ->
val a = row.selectFirst("td.bg_style1")?.selectFirst("a")
?: return@mapNotNull null
val href = a.attrAsRelativeUrl("href")
val title = a.selectFirst("h2")?.text().orEmpty()
val info = row.selectFirst("td[width=100%]") ?: return@mapNotNull null
Manga(
id = generateUid(href),
url = href,
title = title.substringAfter(" / "),
altTitle = title.substringBefore(" / ", "")
.takeUnless { it.isBlank() },
author = info.getElementsContainingOwnText("Автор:").firstOrNull()
?.nextElementSibling()?.ownText(),
coverUrl = row.selectFirst("img.news_pic2")?.absUrl("data-src")
.orEmpty(),
tags = row.selectFirst("span.tag-links")?.select("a")
?.mapToSet {
MangaTag(
title = it.text().toTitleCase(),
key = it.attr("href").substringAfterLast('/'),
source = source,
)
}.orEmpty(),
source = source,
publicUrl = a.absUrl("href"),
rating = RATING_UNKNOWN,
isNsfw = true,
description = row.selectFirst("div.description")?.html(),
state = null,
)
}
}
override suspend fun getDetails(manga: Manga): Manga {
val body = context.httpGet(manga.url.toAbsoluteUrl(getDomain())).parseHtml().body()
val root = body.selectFirst("table.shoutbox")
?: body.parseFailed("Cannot find root")
val info = root.select("div.tbl2")
val lastInfo = info.last()
return manga.copy(
largeCoverUrl = body.selectFirst("img.news_pic2")?.absUrl("src"),
description = info.select("div.blockquote").lastOrNull()?.html() ?: manga.description,
tags = info.select("span.tag-links").firstOrNull()?.select("a")?.mapToSet {
MangaTag(
title = it.text().toTitleCase(),
key = it.attr("href").substringAfterLast('/'),
source = source,
)
}?.plus(manga.tags) ?: manga.tags,
author = lastInfo?.getElementsByAttributeValueContaining("href", "mangaka/")?.text()
?: manga.author,
chapters = listOf(
MangaChapter(
id = manga.id,
url = getReadLink(manga.url),
source = source,
number = 1,
name = manga.title,
scanlator = lastInfo?.getElementsByAttributeValueContaining("href", "perevod/")?.text(),
uploadDate = lastInfo?.getElementsContainingOwnText("Дата:")
?.firstOrNull()
?.html()
?.parseDate() ?: 0L,
branch = null,
),
),
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(getDomain())
val doc = context.httpGet(fullUrl).parseHtml()
val mangaId = chapter.url.substringAfterLast('/').substringBefore('-').toIntOrNull()
val script = doc.select("script").firstNotNullOfOrNull {
it.html().takeIf { x -> x.contains(" images = new ") }
} ?: if (isAuthorized) {
doc.parseFailed("Cannot find pages list")
} else {
throw AuthRequiredException(source)
}
val pagesRegex = Regex("images\\[(\\d+)].src\\s*=\\s*'([^']+)'", RegexOption.MULTILINE)
return pagesRegex.findAll(script).map { match ->
val i = match.groupValues[1].toInt()
val url = match.groupValues[2]
MangaPage(
id = generateUid(url),
url = url,
referer = fullUrl,
preview = if (i <= MAX_THUMB_INDEX && mangaId != null) {
val part2 = url.substringBeforeLast('/')
val part3 = url.substringAfterLast('/')
val part1 = part2.substringBeforeLast('/')
"$part1/thumb/$mangaId/thumb_$part3"
} else {
null
},
source = source,
)
}.toList()
}
override suspend fun getTags(): Set<MangaTag> {
val domain = getDomain()
val doc = context.httpGet("https://$domain/all_manga").parseHtml()
val root = doc.body().getElementsContainingOwnText("Поиск манги по тегам")
.firstOrNull()?.parents()?.find { it.tag().normalName() == "tbody" }
?.selectFirst("td.textbox")?.selectFirst("td.small")
?: doc.parseFailed("Tags root not found")
return root.select("a").mapToSet {
MangaTag(
title = it.text().toTitleCase(),
key = it.attr("href").substringAfterLast('/')
.removeSuffix("+"),
source = source,
)
}
}
override suspend fun getUsername(): String {
val body = context.httpGet("https://${getDomain()}/").parseHtml()
.body()
return body
.getElementsContainingOwnText("Профиль")
.firstOrNull()
?.attr("href")
?.substringAfterLast('/')
?: run {
throw if (body.selectFirst("form[name=\"loginform\"]") != null) {
AuthRequiredException(source)
} else {
body.parseFailed("Cannot find username")
}
}
}
override fun getFaviconUrl(): String {
return "https://${getDomain()}/favicon.jpg"
}
private fun getSortKey(sortOrder: SortOrder) =
when (sortOrder) {
SortOrder.POPULARITY -> "views"
SortOrder.NEWEST -> "date"
SortOrder.RATING -> "like"
else -> "like"
}
private fun String.parseDate(): Long {
val dateString = substringBetweenFirst("Дата:", "<")?.trim() ?: return 0
val dateFormat = SimpleDateFormat("d MMMM yyyy", Locale("ru"))
return dateFormat.tryParse(dateString)
}
private fun getReadLink(url: String): String {
val prefix = url.substringBefore('-', "")
val suffix = url.substringAfter('-').trimStart('-')
return "$prefix-online-$suffix"
}
}

View File

@@ -1,177 +0,0 @@
package org.koitharu.kotatsu.parsers.site
import org.jsoup.nodes.Element
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import org.koitharu.kotatsu.parsers.util.json.getStringOrNull
import org.koitharu.kotatsu.parsers.util.json.mapJSON
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("UNION_MANGAS", "Union Mangás", "pt")
class UnionMangasParser(override val context: MangaLoaderContext) : PagedMangaParser(MangaSource.UNION_MANGAS, 40) {
override val sortOrders = EnumSet.of(
SortOrder.ALPHABETICAL,
SortOrder.POPULARITY,
)
override val configKeyDomain = ConfigKey.Domain("unionleitor.top", emptyArray())
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
if (!query.isNullOrEmpty()) {
return if (page == searchPaginator.firstPage) {
search(query)
} else {
emptyList()
}
}
val tag = tags.oneOrThrowIfMany()
val url = urlBuilder()
.addPathSegment("lista-mangas")
.addPathSegment(
when {
tag != null -> tag.key
sortOrder == SortOrder.ALPHABETICAL -> "a-z"
else -> "visualizacoes"
},
).addPathSegment(page.toString())
val doc = context.httpGet(url.build()).parseHtml()
val root = doc.selectFirstOrThrow("div.tamanho-bloco-perfil")
return root.select(".lista-mangas-novos").map { div ->
val a = div.selectFirstOrThrow("a")
val img = div.selectFirstOrThrow("img")
val href = a.attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = a.attrAsAbsoluteUrl("href"),
title = div.selectLastOrThrow("a").text(),
coverUrl = img.attrAsAbsoluteUrl("src"),
altTitle = null,
rating = RATING_UNKNOWN,
tags = emptySet(),
description = div.selectLast("div")?.ownText(),
state = null,
author = null,
isNsfw = false,
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga {
val doc = context.httpGet(manga.url.toAbsoluteUrl(getDomain())).parseHtml()
val root = doc.selectFirstOrThrow(".perfil-manga")
val dateFormat = SimpleDateFormat("dd/MM/yyyy", Locale.ROOT)
return manga.copy(
rating = root.select("h2")
.find { it.ownText().startsWith('#') }
?.ownText()?.drop(1)?.toFloatOrNull()?.div(10f) ?: manga.rating,
largeCoverUrl = root.selectFirst("img.img-thumbnail")?.attrAsAbsoluteUrlOrNull("src"),
description = root.selectFirst(".panel-default")?.selectFirst(".panel-body")?.html(),
author = root.tableValue("Autor")?.ownText(),
altTitle = root.tableValue("Título(s) Alternativo(s)")?.ownText(),
state = when (root.tableValue("Status")?.selectLast(".label")?.text()) {
"Completo" -> MangaState.FINISHED
"Ativo" -> MangaState.ONGOING
else -> null
},
tags = root.tableValue("Gênero(s)")?.select("a")?.mapToSet {
it.toMangaTag()
} ?: manga.tags,
isNsfw = root.selectFirst(".alert-danger")?.html()?.contains("18 anos") == true,
chapters = root.select("div.row.capitulos").asReversed().mapChapters { i, div ->
val a = div.selectFirstOrThrow("a")
val href = a.attrAsRelativeUrl("href")
val title = a.text()
MangaChapter(
id = generateUid(href),
name = title,
number = i + 1,
url = href,
scanlator = div.selectLast("a")?.text()?.takeUnless { it == title },
uploadDate = dateFormat.tryParse(
a.nextElementSibling()?.text()?.removeSurrounding('(', ')'),
),
branch = null,
source = source,
)
},
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(getDomain())
val doc = context.httpGet(fullUrl).parseHtml()
val root = doc.body().selectFirstOrThrow("article")
return root.selectOrThrow("img[pag]").mapNotNull { img ->
val href = img.attrAsRelativeUrl("src")
if (href.startsWith("/images/banner")) {
return@mapNotNull null
}
MangaPage(
id = generateUid(href),
url = href,
referer = fullUrl,
preview = null,
source = source,
)
}
}
override suspend fun getTags(): Set<MangaTag> {
val doc = context.httpGet(urlBuilder().addPathSegment("lista-mangas").build()).parseHtml()
val ul = doc.body().selectFirstOrThrow(".nav-tabs").selectFirstOrThrow("ul.dropdown-menu")
return ul.select("li").mapToSet { li ->
li.selectFirstOrThrow("a").toMangaTag()
}
}
private suspend fun search(query: String): List<Manga> {
val domain = getDomain()
val json = context.httpGet(
urlBuilder()
.addPathSegments("assets/busca.php")
.addQueryParameter("nomeManga", query)
.build(),
).parseJson()
return json.getJSONArray("items").mapJSON { jo ->
val href = "/pagina-manga/" + jo.getString("url")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(domain),
title = jo.getString("titulo"),
rating = RATING_UNKNOWN,
tags = emptySet(),
author = jo.getStringOrNull("autor"),
coverUrl = jo.getString("imagem"),
state = null,
isNsfw = false,
altTitle = null,
source = source,
)
}
}
private fun Element.tableValue(title: String): Element? {
return select("h4.media-heading")
.find { it.selectFirst("label.subtit-manga")?.text()?.contains(title, ignoreCase = true) == true }
}
private fun Element.toMangaTag() = MangaTag(
title = text().toTitleCase(sourceLocale ?: Locale.ROOT),
key = attr("href").removeSuffix('/').substringAfterLast('/'),
source = source,
)
}

View File

@@ -1,4 +1,4 @@
package org.koitharu.kotatsu.parsers.site
package org.koitharu.kotatsu.parsers.site.all
import androidx.collection.ArraySet
import org.json.JSONArray
@@ -19,13 +19,14 @@ import javax.crypto.spec.IvParameterSpec
import javax.crypto.spec.SecretKeySpec
@MangaSourceParser("BATOTO", "Bato.To")
internal class BatoToParser(override val context: MangaLoaderContext) : PagedMangaParser(
internal class BatoToParser(context: MangaLoaderContext) : PagedMangaParser(
context = context,
source = MangaSource.BATOTO,
pageSize = 60,
searchPageSize = 20,
) {
override val sortOrders: Set<SortOrder> = EnumSet.of(
override val availableSortOrders: Set<SortOrder> = EnumSet.of(
SortOrder.NEWEST,
SortOrder.UPDATED,
SortOrder.POPULARITY,
@@ -34,7 +35,30 @@ internal class BatoToParser(override val context: MangaLoaderContext) : PagedMan
override val configKeyDomain = ConfigKey.Domain(
"bato.to",
arrayOf("bato.to", "mto.to", "mangatoto.com", "battwo.com", "batotwo.com", "comiko.net", "batotoo.com"),
"batocomic.com",
"batocomic.net",
"batocomic.org",
"batotoo.com",
"batotwo.com",
"battwo.com",
"comiko.net",
"comiko.org",
"mangatoto.com",
"mangatoto.net",
"mangatoto.org",
"readtoto.com",
"readtoto.net",
"readtoto.org",
"dto.to",
"hto.to",
"mto.to",
"wto.to",
"xbato.com",
"xbato.net",
"xbato.org",
"zbato.com",
"zbato.net",
"zbato.org",
)
override suspend fun getListPage(
@@ -49,7 +73,7 @@ internal class BatoToParser(override val context: MangaLoaderContext) : PagedMan
@Suppress("NON_EXHAUSTIVE_WHEN_STATEMENT")
val url = buildString {
append("https://")
append(getDomain())
append(domain)
append("/browse?sort=")
when (sortOrder) {
SortOrder.UPDATED,
@@ -58,6 +82,7 @@ internal class BatoToParser(override val context: MangaLoaderContext) : PagedMan
SortOrder.POPULARITY -> append("views_a.za")
SortOrder.NEWEST -> append("create.za")
SortOrder.ALPHABETICAL -> append("title.az")
SortOrder.RATING -> Unit
}
if (!tags.isNullOrEmpty()) {
append("&genres=")
@@ -70,7 +95,7 @@ internal class BatoToParser(override val context: MangaLoaderContext) : PagedMan
}
override suspend fun getDetails(manga: Manga): Manga {
val root = context.httpGet(manga.url.toAbsoluteUrl(getDomain())).parseHtml()
val root = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
.requireElementById("mainer")
val details = root.selectFirstOrThrow(".detail-set")
val attrs = details.selectFirst(".attr-main")?.select(".attr-item")?.associate {
@@ -93,16 +118,15 @@ internal class BatoToParser(override val context: MangaLoaderContext) : PagedMan
chapters = root.selectFirst(".episode-list")
?.selectFirst(".main")
?.children()
?.reversed()
?.mapChapters { i, div ->
?.mapChapters(reversed = true) { i, div ->
div.parseChapter(i)
}.orEmpty(),
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(getDomain())
val scripts = context.httpGet(fullUrl).parseHtml().select("script")
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val scripts = webClient.httpGet(fullUrl).parseHtml().select("script")
for (script in scripts) {
val scriptSrc = script.html()
val p = scriptSrc.indexOf("const imgHttpLis =")
@@ -126,7 +150,6 @@ internal class BatoToParser(override val context: MangaLoaderContext) : PagedMan
result += MangaPage(
id = generateUid(url),
url = url + "?" + args.getString(i),
referer = fullUrl,
preview = null,
source = source,
)
@@ -136,9 +159,9 @@ internal class BatoToParser(override val context: MangaLoaderContext) : PagedMan
throw ParseException("Cannot find images list", fullUrl)
}
override suspend fun getTags(): Set<MangaTag> {
val scripts = context.httpGet(
"https://${getDomain()}/browse",
override suspend fun getAvailableTags(): Set<MangaTag> {
val scripts = webClient.httpGet(
"https://${domain}/browse",
).parseHtml().selectOrThrow("script")
for (script in scripts) {
val genres = script.html().substringBetweenFirst("const _genres =", ";") ?: continue
@@ -157,12 +180,10 @@ internal class BatoToParser(override val context: MangaLoaderContext) : PagedMan
throw ParseException("Cannot find gernes list", scripts[0].baseUri())
}
override fun getFaviconUrl(): String = "https://styles.amarkcdn.com/img/batoto/favicon.ico?v0"
private suspend fun search(page: Int, query: String): List<Manga> {
val url = buildString {
append("https://")
append(getDomain())
append(domain)
append("/search?word=")
append(query.replace(' ', '+'))
append("&page=")
@@ -177,7 +198,7 @@ internal class BatoToParser(override val context: MangaLoaderContext) : PagedMan
?.toIntOrNull() ?: body.parseFailed("Cannot determine current page")
private suspend fun parseList(url: String, page: Int): List<Manga> {
val body = context.httpGet(url).parseHtml().body()
val body = webClient.httpGet(url).parseHtml().body()
if (body.selectFirst(".browse-no-matches") != null) {
return emptyList()
}
@@ -307,4 +328,4 @@ internal class BatoToParser(override val context: MangaLoaderContext) : PagedMan
} else byteArrayOf(),
)
}
}
}

View File

@@ -1,8 +1,7 @@
package org.koitharu.kotatsu.parsers.site
package org.koitharu.kotatsu.parsers.site.all
import androidx.collection.ArraySet
import androidx.collection.SparseArrayCompat
import org.json.JSONArray
import org.json.JSONObject
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaParser
@@ -22,11 +21,11 @@ private const val PAGE_SIZE = 20
private const val CHAPTERS_LIMIT = 99999
@MangaSourceParser("COMICK_FUN", "ComicK")
internal class ComickFunParser(override val context: MangaLoaderContext) : MangaParser(MangaSource.COMICK_FUN) {
internal class ComickFunParser(context: MangaLoaderContext) : MangaParser(context, MangaSource.COMICK_FUN) {
override val configKeyDomain = ConfigKey.Domain("comick.fun", null)
override val configKeyDomain = ConfigKey.Domain("comick.app")
override val sortOrders: Set<SortOrder> = EnumSet.of(
override val availableSortOrders: Set<SortOrder> = EnumSet.of(
SortOrder.POPULARITY,
SortOrder.UPDATED,
SortOrder.RATING,
@@ -41,11 +40,11 @@ internal class ComickFunParser(override val context: MangaLoaderContext) : Manga
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val domain = getDomain()
val domain = domain
val url = buildString {
append("https://api.")
append(domain)
append("/search?tachiyomi=true")
append("/v1.0/search?tachiyomi=true")
if (!query.isNullOrEmpty()) {
if (offset > 0) {
return emptyList()
@@ -71,7 +70,7 @@ internal class ComickFunParser(override val context: MangaLoaderContext) : Manga
)
}
}
val ja = context.httpGet(url).parseJsonArray()
val ja = webClient.httpGet(url).parseJsonArray()
val tagsMap = cachedTags ?: loadTags()
return ja.mapJSON { jo ->
val slug = jo.getString("slug")
@@ -86,7 +85,7 @@ internal class ComickFunParser(override val context: MangaLoaderContext) : Manga
coverUrl = jo.getString("cover_url"),
largeCoverUrl = null,
description = jo.getStringOrNull("desc"),
tags = jo.selectGenres("genres", tagsMap),
tags = jo.selectGenres(tagsMap),
state = runCatching {
if (jo.getBoolean("translation_completed")) {
MangaState.FINISHED
@@ -101,45 +100,44 @@ internal class ComickFunParser(override val context: MangaLoaderContext) : Manga
}
override suspend fun getDetails(manga: Manga): Manga {
val domain = getDomain()
val domain = domain
val url = "https://api.$domain/comic/${manga.url}?tachiyomi=true"
val jo = context.httpGet(url).parseJson()
val jo = webClient.httpGet(url).parseJson()
val comic = jo.getJSONObject("comic")
return manga.copy(
title = comic.getString("title"),
altTitle = null, // TODO
isNsfw = jo.getBoolean("matureContent") || comic.getBoolean("hentai"),
description = comic.getStringOrNull("parsed") ?: comic.getString("desc"),
tags = manga.tags + jo.getJSONArray("genres").mapJSONToSet {
tags = manga.tags + comic.getJSONArray("md_comic_md_genres").mapJSONToSet {
val g = it.getJSONObject("md_genres")
MangaTag(
title = it.getString("name"),
key = it.getString("slug"),
title = g.getString("name"),
key = g.getString("slug"),
source = source,
)
},
author = jo.getJSONArray("artists").optJSONObject(0)?.getString("name"),
chapters = getChapters(comic.getLong("id")),
chapters = getChapters(comic.getString("hid")),
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val jo = context.httpGet(
"https://api.${getDomain()}/chapter/${chapter.url}?tachiyomi=true",
val jo = webClient.httpGet(
"https://api.${domain}/chapter/${chapter.url}?tachiyomi=true",
).parseJson().getJSONObject("chapter")
val referer = "https://${getDomain()}/"
return jo.getJSONArray("images").mapJSON {
val url = it.getString("url")
MangaPage(
id = generateUid(url),
url = url,
referer = referer,
preview = null,
source = source,
)
}
}
override suspend fun getTags(): Set<MangaTag> {
override suspend fun getAvailableTags(): Set<MangaTag> {
val sparseArray = cachedTags ?: loadTags()
val set = ArraySet<MangaTag>(sparseArray.size())
for (i in 0 until sparseArray.size()) {
@@ -149,7 +147,7 @@ internal class ComickFunParser(override val context: MangaLoaderContext) : Manga
}
private suspend fun loadTags(): SparseArrayCompat<MangaTag> {
val ja = context.httpGet("https://api.${getDomain()}/genre").parseJsonArray()
val ja = webClient.httpGet("https://api.${domain}/genre").parseJsonArray()
val tags = SparseArrayCompat<MangaTag>(ja.length())
for (jo in ja.JSONIterator()) {
tags.append(
@@ -165,46 +163,49 @@ internal class ComickFunParser(override val context: MangaLoaderContext) : Manga
return tags
}
private suspend fun getChapters(id: Long): List<MangaChapter> {
val ja = context.httpGet(
url = "https://api.${getDomain()}/comic/$id/chapter?tachiyomi=true&limit=$CHAPTERS_LIMIT",
private suspend fun getChapters(hid: String): List<MangaChapter> {
val ja = webClient.httpGet(
url = "https://api.${domain}/comic/$hid/chapters?limit=$CHAPTERS_LIMIT",
).parseJson().getJSONArray("chapters")
val dateFormat = SimpleDateFormat("yyyy-MM-dd")
val counters = HashMap<Locale, Int>()
return ja.mapReversed { jo ->
val list = ja.toJSONList().reversed()
val chaptersBuilder = ChaptersListBuilder(list.size)
val branchedChapters = HashMap<String?, HashMap<Pair<String?, String?>, MangaChapter>>()
for (jo in list) {
val vol = jo.getStringOrNull("vol")
val chap = jo.getStringOrNull("chap")
val volChap = vol to chap
val locale = Locale.forLanguageTag(jo.getString("lang"))
var number = counters[locale] ?: 0
number++
counters[locale] = number
MangaChapter(
val lc = locale.getDisplayName(locale).toTitleCase(locale)
val branch = (list.indices).firstNotNullOf { i ->
val b = if (i == 0) lc else "$lc ($i)"
if (branchedChapters[b]?.get(volChap) == null) b else null
}
val chapter = MangaChapter(
id = generateUid(jo.getLong("id")),
name = buildString {
jo.getStringOrNull("vol")?.let { append("Vol ").append(it).append(' ') }
jo.getStringOrNull("chap")?.let { append("Chap ").append(it) }
vol?.let { append("Vol ").append(it).append(' ') }
chap?.let { append("Chap ").append(it) }
jo.getStringOrNull("title")?.let { append(": ").append(it) }
},
number = number,
number = branchedChapters[branch]?.size?.plus(1) ?: 1,
url = jo.getString("hid"),
scanlator = jo.optJSONArray("group_name")?.optString(0),
scanlator = jo.optJSONArray("group_name")?.asIterable<String>()?.joinToString()
?.takeUnless { it.isBlank() },
uploadDate = dateFormat.tryParse(jo.getString("created_at").substringBefore('T')),
branch = locale.getDisplayName(locale).toTitleCase(locale),
branch = branch,
source = source,
)
if (chaptersBuilder.add(chapter)) {
branchedChapters.getOrPut(branch, ::HashMap)[volChap] = chapter
}
}
return chaptersBuilder.toList()
}
private inline fun <R> JSONArray.mapReversed(block: (JSONObject) -> R): List<R> {
val len = length()
val destination = ArrayList<R>(len)
for (i in (0 until len).reversed()) {
val jo = getJSONObject(i)
destination.add(block(jo))
}
return destination
}
private fun JSONObject.selectGenres(name: String, tags: SparseArrayCompat<MangaTag>): Set<MangaTag> {
val array = optJSONArray(name) ?: return emptySet()
private fun JSONObject.selectGenres(tags: SparseArrayCompat<MangaTag>): Set<MangaTag> {
val array = optJSONArray("genres") ?: return emptySet()
val res = ArraySet<MangaTag>(array.length())
for (i in 0 until array.length()) {
val id = array.getInt(i)
@@ -213,4 +214,4 @@ internal class ComickFunParser(override val context: MangaLoaderContext) : Manga
}
return res
}
}
}

View File

@@ -1,5 +1,8 @@
package org.koitharu.kotatsu.parsers.site
package org.koitharu.kotatsu.parsers.site.all
import androidx.collection.SparseArrayCompat
import androidx.collection.set
import okhttp3.HttpUrl.Companion.toHttpUrlOrNull
import org.jsoup.nodes.Element
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaParserAuthProvider
@@ -15,24 +18,29 @@ import kotlin.math.pow
private const val DOMAIN_UNAUTHORIZED = "e-hentai.org"
private const val DOMAIN_AUTHORIZED = "exhentai.org"
@MangaSourceParser("EXHENTAI", "ExHentai")
@MangaSourceParser("EXHENTAI", "ExHentai", type = ContentType.HENTAI)
internal class ExHentaiParser(
override val context: MangaLoaderContext,
) : PagedMangaParser(MangaSource.EXHENTAI, pageSize = 25), MangaParserAuthProvider {
context: MangaLoaderContext,
) : PagedMangaParser(context, MangaSource.EXHENTAI, pageSize = 25), MangaParserAuthProvider {
override val sortOrders: Set<SortOrder> = Collections.singleton(
override val availableSortOrders: Set<SortOrder> = Collections.singleton(
SortOrder.NEWEST,
)
override val configKeyDomain: ConfigKey.Domain
get() = ConfigKey.Domain(if (isAuthorized) DOMAIN_AUTHORIZED else DOMAIN_UNAUTHORIZED, null)
get() = ConfigKey.Domain(
if (isAuthorized) DOMAIN_AUTHORIZED else DOMAIN_UNAUTHORIZED,
if (isAuthorized) DOMAIN_UNAUTHORIZED else DOMAIN_AUTHORIZED,
)
override val authUrl: String
get() = "https://${getDomain()}/bounce_login.php"
get() = "https://${domain}/bounce_login.php"
private val ratingPattern = Regex("-?[0-9]+px")
private val authCookies = arrayOf("ipb_member_id", "ipb_pass_hash")
private var updateDm = false
private val nextPages = SparseArrayCompat<Long>()
private val suspiciousContentKey = ConfigKey.ShowSuspiciousContent(true)
override val isAuthorized: Boolean
get() {
@@ -54,6 +62,7 @@ internal class ExHentaiParser(
init {
context.cookieJar.insertCookies(DOMAIN_AUTHORIZED, "nw=1", "sl=dm_2")
context.cookieJar.insertCookies(DOMAIN_UNAUTHORIZED, "nw=1", "sl=dm_2")
paginator.firstPage = 0
}
override suspend fun getListPage(
@@ -63,11 +72,16 @@ internal class ExHentaiParser(
sortOrder: SortOrder,
): List<Manga> {
var search = query?.urlEncoded().orEmpty()
val next = nextPages.get(page, 0L)
if (page > 0 && next == 0L) {
assert(false) { "Page timestamp not found" }
return emptyList()
}
val url = buildString {
append("https://")
append(getDomain())
append("/?page=")
append(page)
append(domain)
append("/?next=")
append(next)
if (!tags.isNullOrEmpty()) {
var fCats = 0
for (tag in tags) {
@@ -88,8 +102,12 @@ internal class ExHentaiParser(
if (updateDm) {
append("&inline_set=dm_e")
}
append("&advsearch=1")
if (config[suspiciousContentKey]) {
append("&f_sh=on")
}
}
val body = context.httpGet(url).parseHtml().body()
val body = webClient.httpGet(url).parseHtml().body()
val root = body.selectFirst("table.itg")
?.selectFirst("tbody")
?: if (updateDm) {
@@ -99,6 +117,7 @@ internal class ExHentaiParser(
return getListPage(page, query, tags, sortOrder)
}
updateDm = false
nextPages[page + 1] = getNextTimestamp(body)
return root.children().mapNotNull { tr ->
if (tr.childrenSize() != 2) return@mapNotNull null
val (td1, td2) = tr.children()
@@ -132,7 +151,7 @@ internal class ExHentaiParser(
}
override suspend fun getDetails(manga: Manga): Manga {
val doc = context.httpGet(manga.url.toAbsoluteUrl(getDomain())).parseHtml()
val doc = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
val root = doc.body().selectFirstOrThrow("div.gm")
val cover = root.getElementById("gd1")?.children()?.first()
val title = root.getElementById("gd2")
@@ -176,14 +195,13 @@ internal class ExHentaiParser(
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val doc = context.httpGet(chapter.url.toAbsoluteUrl(getDomain())).parseHtml()
val doc = webClient.httpGet(chapter.url.toAbsoluteUrl(domain)).parseHtml()
val root = doc.body().requireElementById("gdt")
return root.select("a").map { a ->
val url = a.attrAsRelativeUrl("href")
MangaPage(
id = generateUid(url),
url = url,
referer = a.absUrl("href"),
preview = null,
source = source,
)
@@ -191,12 +209,12 @@ internal class ExHentaiParser(
}
override suspend fun getPageUrl(page: MangaPage): String {
val doc = context.httpGet(page.url.toAbsoluteUrl(getDomain())).parseHtml()
val doc = webClient.httpGet(page.url.toAbsoluteUrl(domain)).parseHtml()
return doc.body().requireElementById("img").attrAsAbsoluteUrl("src")
}
override suspend fun getTags(): Set<MangaTag> {
val doc = context.httpGet("https://${getDomain()}").parseHtml()
override suspend fun getAvailableTags(): Set<MangaTag> {
val doc = webClient.httpGet("https://${domain}").parseHtml()
val root = doc.body().requireElementById("searchbox").selectFirstOrThrow("table")
return root.select("div.cs").mapNotNullToSet { div ->
val id = div.id().substringAfterLast('_').toIntOrNull()
@@ -210,7 +228,7 @@ internal class ExHentaiParser(
}
override suspend fun getUsername(): String {
val doc = context.httpGet("https://forums.$DOMAIN_UNAUTHORIZED/").parseHtml().body()
val doc = webClient.httpGet("https://forums.$DOMAIN_UNAUTHORIZED/").parseHtml().body()
val username = doc.getElementById("userlinks")
?.getElementsByAttributeValueContaining("href", "showuser=")
?.firstOrNull()
@@ -223,6 +241,11 @@ internal class ExHentaiParser(
return username
}
override fun onCreateConfig(keys: MutableCollection<ConfigKey<*>>) {
super.onCreateConfig(keys)
keys.add(suspiciousContentKey)
}
private fun isAuthorized(domain: String): Boolean {
val cookies = context.cookieJar.getCookies(domain).mapToSet { x -> x.name }
return authCookies.all { it in cookies }
@@ -276,4 +299,12 @@ internal class ExHentaiParser(
val num = className.drop(2).toIntOrNull(16) ?: return null
return 2.0.pow(num).toInt().toString()
}
}
private fun getNextTimestamp(root: Element): Long {
return root.getElementById("unext")
?.attrAsAbsoluteUrlOrNull("href")
?.toHttpUrlOrNull()
?.queryParameter("next")
?.toLongOrNull() ?: 1
}
}

View File

@@ -0,0 +1,182 @@
package org.koitharu.kotatsu.parsers.site.all
import androidx.collection.ArraySet
import kotlinx.coroutines.async
import kotlinx.coroutines.awaitAll
import kotlinx.coroutines.coroutineScope
import org.jsoup.nodes.Element
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.util.*
@MangaSourceParser("IMHENTAI", "ImHentai", type = ContentType.HENTAI)
internal class ImHentai(context: MangaLoaderContext) :
PagedMangaParser(context, MangaSource.IMHENTAI, pageSize = 20) {
override val availableSortOrders: Set<SortOrder> =
EnumSet.of(SortOrder.UPDATED, SortOrder.POPULARITY, SortOrder.RATING)
override val configKeyDomain = ConfigKey.Domain("imhentai.xxx")
override val isMultipleTagsSupported = false
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val tag = tags.oneOrThrowIfMany()
val url = buildString {
append("https://")
append(domain)
if (!query.isNullOrEmpty()) {
append("/search/?key=")
append(query.urlEncoded())
append("&page=")
append(page)
} else if (!tags.isNullOrEmpty()) {
append("/tag/")
append(tag?.key.orEmpty())
append("/")
when (sortOrder) {
SortOrder.UPDATED -> append("")
SortOrder.POPULARITY -> append("popular/")
else -> append("")
}
append("?page=")
append(page)
} else {
append("/search/?page=")
append(page)
when (sortOrder) {
SortOrder.UPDATED -> append("&lt=1&pp=0")
SortOrder.POPULARITY -> append("&lt=0&pp=1")
SortOrder.RATING -> append("&lt=0&pp=0")
else -> append("&lt=1&pp=0")
}
}
}
val doc = webClient.httpGet(url).parseHtml()
return doc.select("div.galleries div.thumb").map { div ->
val a = div.selectFirstOrThrow(".inner_thumb a")
val href = a.attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(domain),
coverUrl = a.selectFirst("img")?.src().orEmpty(),
title = div.selectFirst(".caption")?.text().orEmpty(),
altTitle = null,
rating = RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = null,
source = source,
isNsfw = isNsfwSource,
)
}
}
//Tags are deliberately reduced because there are too many and this slows down the application.
//only the most popular ones are taken.
override suspend fun getAvailableTags(): Set<MangaTag> {
return coroutineScope {
(1..3).map { page ->
async { getTags(page) }
}
}.awaitAll().flattenTo(ArraySet(360))
}
private suspend fun getTags(page: Int): Set<MangaTag> {
val url = "https://$domain/tags/popular/?page=$page"
val root = webClient.httpGet(url).parseHtml()
return root.parseTags()
}
private fun Element.parseTags() = select("div.stags a.tag_btn").mapToSet {
val href = it.attr("href").substringAfterLast("tag/").substringBeforeLast('/')
MangaTag(
key = href,
title = it.selectFirstOrThrow("h3.list_tag").text(),
source = source,
)
}
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val fullUrl = manga.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
manga.copy(
tags = doc.body().select("li:contains(Tags) a.tag").mapNotNullToSet {
val href = it.attr("href").substringAfterLast("tag/").substringBeforeLast('/')
val name = it.html().substringBeforeLast("<span")
MangaTag(
key = href,
title = name,
source = source,
)
},
author = doc.selectFirst("li:contains(Artists) a.tag")?.html()?.substringBefore("<span"),
chapters = listOf(
MangaChapter(
id = manga.id,
name = manga.title,
number = 1,
url = manga.url,
scanlator = null,
uploadDate = 0,
branch = doc.selectFirst("li:contains(Language) a.tag")?.html()?.substringBeforeLast("<span"),
source = source,
),
),
)
}
override suspend fun getRelatedManga(seed: Manga): List<Manga> {
val doc = webClient.httpGet(seed.url.toAbsoluteUrl(domain)).parseHtml()
val root = doc.body().selectFirstOrThrow("div.related")
return root.select("div.thumb").mapNotNull { div ->
val a = div.selectFirstOrThrow(".inner_thumb a")
val href = a.attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(domain),
coverUrl = a.selectFirst("img")?.src().orEmpty(),
title = div.selectFirst(".caption")?.text().orEmpty(),
altTitle = null,
rating = RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = null,
source = source,
isNsfw = false,
)
}
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
val totalPages = doc.selectFirstOrThrow(".pages").text().replace("Pages: ", "").toInt() + 1
val domainImg = doc.requireElementById("append_thumbs").selectFirstOrThrow("img").src()?.replace("1t.jpg", "")
val pages = ArrayList<MangaPage>(totalPages)
for (i in 1 until totalPages) {
val url = "$domainImg$i.jpg"
pages.add(
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
),
)
}
return pages
}
}

View File

@@ -0,0 +1,320 @@
package org.koitharu.kotatsu.parsers.site.all
import kotlinx.coroutines.async
import kotlinx.coroutines.coroutineScope
import okhttp3.Headers
import okhttp3.HttpUrl
import okhttp3.HttpUrl.Companion.toHttpUrl
import org.json.JSONObject
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaParser
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.exception.NotFoundException
import org.koitharu.kotatsu.parsers.exception.ParseException
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import org.koitharu.kotatsu.parsers.util.json.*
import java.util.*
import javax.crypto.Mac
import javax.crypto.spec.SecretKeySpec
internal abstract class LineWebtoonsParser(
context: MangaLoaderContext,
source: MangaSource,
) : MangaParser(context, source) {
override val isMultipleTagsSupported = false
private val signer by lazy {
WebtoonsUrlSigner("gUtPzJFZch4ZyAGviiyH94P99lQ3pFdRTwpJWDlSGFfwgpr6ses5ALOxWHOIT7R1")
}
// we don't __really__ support changing this domain because:
// 1. I don't think other websites have this exact API
// 2. most communication is done with other domains (hosting API and static content), which are not configurable
// 3. we rely on the HTTP client setting the referer header to webtoons.com
//
// This effectively means that changing the domain will break the source. Yikes
override val configKeyDomain = ConfigKey.Domain("webtoons.com")
private val apiDomain = "global.apis.naver.com"
private val staticDomain = "webtoon-phinf.pstatic.net"
override val availableSortOrders: Set<SortOrder> = EnumSet.of(
SortOrder.POPULARITY,
// doesn't actually sort by rating, but by likes
// this should be fine though
SortOrder.RATING,
SortOrder.UPDATED,
)
override val headers: Headers
get() = Headers.Builder()
.add("User-Agent", "nApps (Android 12;; linewebtoon; 3.1.0)")
.build()
override suspend fun getPageUrl(page: MangaPage): String {
return page.url.toAbsoluteUrl(staticDomain)
}
// some language tags do not map perfectly to the ones used by the API
private val languageCode: String
get() = when (val tag = sourceLocale.toLanguageTag()) {
"in" -> "id"
"zh" -> "zh-hant"
else -> tag
}
private suspend fun getChapters(titleNo: Long): List<MangaChapter> {
val firstResult = makeRequest(
url = "/lineWebtoon/webtoon/challengeEpisodeList.json?v=2&titleNo=$titleNo&startIndex=0&pageSize=30",
)
val totalEpisodeCount = firstResult
.getJSONObject("episodeList")
.getInt("totalServiceEpisodeCount")
val episodes = firstResult
.getJSONObject("episodeList")
.getJSONArray("episode")
.toJSONList()
.toMutableList()
while (episodes.count() < totalEpisodeCount) {
val page = makeRequest(
url = "/lineWebtoon/webtoon/challengeEpisodeList.json?v=2&titleNo=$titleNo&startIndex=${episodes.count()}&pageSize=30",
).getJSONObject("episodeList")
.getJSONArray("episode")
.toJSONList()
episodes.addAll(page)
}
return episodes.mapChapters { i, jo ->
MangaChapter(
id = generateUid("$titleNo-$i"),
name = jo.getString("episodeTitle"),
number = jo.getInt("episodeSeq"),
url = "$titleNo-${jo.get("episodeNo")}",
uploadDate = jo.getLong("modifyYmdt"),
branch = null,
scanlator = null,
source = source,
)
}.sortedBy { it.number }
}
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val titleNo = manga.url.toLong()
val chaptersDeferred = async { getChapters(titleNo) }
makeRequest("/lineWebtoon/webtoon/challengeTitleInfo.json?v=2&titleNo=${titleNo}")
.getJSONObject("titleInfo")
.let { jo ->
Manga(
id = generateUid(titleNo),
title = jo.getString("title"),
altTitle = null,
url = "$titleNo",
publicUrl = "https://$domain/$languageCode/canvas/a/list?title_no=${titleNo}",
rating = jo.getFloatOrDefault("starScoreAverage", -10f) / 10f,
isNsfw = jo.getBooleanOrDefault("ageGradeNotice", isNsfwSource),
coverUrl = jo.getString("thumbnail").toAbsoluteUrl(staticDomain),
largeCoverUrl = jo.getStringOrNull("thumbnailVertical")?.toAbsoluteUrl(staticDomain),
tags = setOf(parseTag(jo.getJSONObject("genreInfo"))),
author = jo.getStringOrNull("writingAuthorName"),
description = jo.getString("synopsis"),
// I don't think the API provides this info
state = null,
chapters = chaptersDeferred.await(),
source = source,
)
}
}
override suspend fun getList(
offset: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val genre = tags.oneOrThrowIfMany()?.key ?: "ALL"
val sortOrderStr = when (sortOrder) {
SortOrder.UPDATED -> "UPDATE"
SortOrder.POPULARITY -> "READ_COUNT"
SortOrder.RATING -> "LIKEIT"
else -> throw IllegalArgumentException("Unsupported sort order: $sortOrder")
}
val manga = if (query != null) {
if (!tags.isNullOrEmpty()) {
throw IllegalArgumentException("This source does not support search with tags")
}
makeRequest("/lineWebtoon/webtoon/searchChallenge?query=${query.urlEncoded()}&startIndex=${offset + 1}&pageSize=20")
.getJSONObject("challengeSearch")
.getJSONArray("titleList")
.mapJSON { jo ->
val titleNo = jo.getLong("titleNo")
Manga(
id = generateUid(titleNo),
title = jo.getString("title"),
altTitle = null,
url = titleNo.toString(),
publicUrl = "https://$domain/$languageCode/canvas/a/list?title_no=$titleNo",
rating = RATING_UNKNOWN,
isNsfw = isNsfwSource,
coverUrl = jo.getString("thumbnail").toAbsoluteUrl(staticDomain),
largeCoverUrl = null,
tags = emptySet(),
author = jo.getStringOrNull("writingAuthorName"),
description = null,
state = null,
source = source,
)
}
} else {
val result =
makeRequest("/lineWebtoon/webtoon/challengeGenreTitleList.json?genre=$genre&sortOrder=$sortOrderStr&startIndex=${offset + 1}&pageSize=20")
val genres = result.getJSONObject("genreList")
.getJSONArray("challengeGenres")
.mapJSON { jo -> parseTag(jo) }
.associateBy { tag -> tag.key }
result
.getJSONObject("titleList")
.getJSONArray("titles")
.mapJSON { jo ->
val titleNo = jo.getLong("titleNo")
Manga(
id = generateUid(titleNo),
title = jo.getString("title"),
altTitle = null,
url = titleNo.toString(),
publicUrl = "https://$domain/$languageCode/canvas/a/list?title_no=$titleNo",
rating = jo.getFloatOrDefault("starScoreAverage", -10f) / 10f,
isNsfw = jo.getBooleanOrDefault("ageGradeNotice", isNsfwSource),
coverUrl = jo.getString("thumbnail").toAbsoluteUrl(staticDomain),
largeCoverUrl = jo.getStringOrNull("thumbnailVertical")?.toAbsoluteUrl(staticDomain),
tags = setOfNotNull(genres[jo.getString("representGenre")]),
author = jo.getStringOrNull("writingAuthorName"),
description = jo.getString("synopsis"),
// I don't think the API provides this info
state = null,
source = source,
)
}
}
return manga
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val (titleNo, episodeNo) = requireNotNull(chapter.url.splitTwoParts('-'))
return makeRequest("/lineWebtoon/webtoon/challengeEpisodeInfo.json?v=2&titleNo=$titleNo&episodeNo=$episodeNo")
.getJSONObject("episodeInfo")
.getJSONArray("imageInfo")
.mapJSONIndexed { i, jo ->
MangaPage(
id = generateUid("$titleNo-$episodeNo-$i"),
url = jo.getString("url"),
preview = null,
source = source,
)
}
}
private fun parseTag(jo: JSONObject): MangaTag {
return MangaTag(
title = jo.getString("name"),
key = jo.getString("code"),
source = source,
)
}
override suspend fun getAvailableTags(): Set<MangaTag> {
return makeRequest("/lineWebtoon/webtoon/challengeGenreList.json")
.getJSONObject("genreList")
.getJSONArray("challengeGenres")
.mapJSONToSet { jo -> parseTag(jo) }
}
private suspend fun makeRequest(url: String): JSONObject {
val resp = webClient.httpGet(finalizeUrl(url))
val message: JSONObject? = resp.parseJson().optJSONObject("message")
return when (resp.code) {
in 200..299 -> checkNotNull(message).getJSONObject("result")
404 -> throw NotFoundException(message?.getStringOrNull("message").orEmpty(), url)
else -> {
val code = message?.getIntOrDefault("code", 0)
val errorMessage = message?.getStringOrNull("message")
throw ParseException("Api error (code=$code): $errorMessage", url)
}
}
}
private fun finalizeUrl(url: String): HttpUrl {
val httpUrl = url.toAbsoluteUrl(apiDomain).toHttpUrl()
val builder = httpUrl.newBuilder()
.addQueryParameter("serviceZone", "GLOBAL")
if (httpUrl.queryParameter("v") == null) {
builder.addQueryParameter("v", "1")
}
builder.addQueryParameter("language", languageCode)
.addQueryParameter("locale", "languageCode")
.addQueryParameter("platform", "APP_ANDROID")
signer.makeEncryptUrl(builder)
return builder.build()
}
@MangaSourceParser("LINEWEBTOONS_EN", "LineWebtoons English", "en", type = ContentType.MANGA)
class English(context: MangaLoaderContext) : LineWebtoonsParser(context, MangaSource.LINEWEBTOONS_EN)
@MangaSourceParser("LINEWEBTOONS_ZH", "LineWebtoons Chinese", "zh", type = ContentType.MANGA)
class Chinese(context: MangaLoaderContext) : LineWebtoonsParser(context, MangaSource.LINEWEBTOONS_ZH)
@MangaSourceParser("LINEWEBTOONS_TH", "LineWebtoons Thai", "th", type = ContentType.MANGA)
class Thai(context: MangaLoaderContext) : LineWebtoonsParser(context, MangaSource.LINEWEBTOONS_TH)
@MangaSourceParser("LINEWEBTOONS_ID", "LineWebtoons Indonesian", "id", type = ContentType.MANGA)
class Indonesian(context: MangaLoaderContext) : LineWebtoonsParser(context, MangaSource.LINEWEBTOONS_ID)
@MangaSourceParser("LINEWEBTOONS_ES", "LineWebtoons Spanish", "es", type = ContentType.MANGA)
class Spanish(context: MangaLoaderContext) : LineWebtoonsParser(context, MangaSource.LINEWEBTOONS_ES)
@MangaSourceParser("LINEWEBTOONS_FR", "LineWebtoons French", "fr", type = ContentType.MANGA)
class French(context: MangaLoaderContext) : LineWebtoonsParser(context, MangaSource.LINEWEBTOONS_FR)
@MangaSourceParser("LINEWEBTOONS_DE", "LineWebtoons German", "de", type = ContentType.MANGA)
class German(context: MangaLoaderContext) : LineWebtoonsParser(context, MangaSource.LINEWEBTOONS_DE)
private inner class WebtoonsUrlSigner(private val secret: String) {
private val mac = Mac.getInstance("HmacSHA1").apply {
this.init(SecretKeySpec(secret.encodeToByteArray(), "HmacSHA1"))
}
private fun getMessage(url: String, msgpad: String): String {
return url.substring(0, 0xFF.coerceAtMost(url.length)) + msgpad
}
private fun getMessageDigest(s: String): String {
val signedMessage = synchronized(mac) { mac.doFinal(s.toByteArray()) }
return context.encodeBase64(signedMessage)
}
fun makeEncryptUrl(urlBuilder: HttpUrl.Builder) {
val msgpad = Calendar.getInstance().timeInMillis.toString()
val digest = getMessageDigest(getMessage(urlBuilder.build().toString(), msgpad))
urlBuilder
.addQueryParameter("msgpad", msgpad)
.addQueryParameter("md", digest)
// .addEncodedQueryParameter("md", digest.urlEncoded())
}
}
}

View File

@@ -1,4 +1,4 @@
package org.koitharu.kotatsu.parsers.site
package org.koitharu.kotatsu.parsers.site.all
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.async
@@ -20,16 +20,17 @@ private const val PAGE_SIZE = 20
private const val CHAPTERS_FIRST_PAGE_SIZE = 120
private const val CHAPTERS_MAX_PAGE_SIZE = 500
private const val CHAPTERS_PARALLELISM = 3
private const val CHAPTERS_MAX_COUNT = 10_000 // strange api behavior, looks like a bug
private const val CONTENT_RATING =
"contentRating[]=safe&contentRating[]=suggestive&contentRating[]=erotica&contentRating[]=pornographic"
private const val LOCALE_FALLBACK = "en"
@MangaSourceParser("MANGADEX", "MangaDex")
internal class MangaDexParser(override val context: MangaLoaderContext) : MangaParser(MangaSource.MANGADEX) {
internal class MangaDexParser(context: MangaLoaderContext) : MangaParser(context, MangaSource.MANGADEX) {
override val configKeyDomain = ConfigKey.Domain("mangadex.org", null)
override val configKeyDomain = ConfigKey.Domain("mangadex.org")
override val sortOrders: EnumSet<SortOrder> = EnumSet.of(
override val availableSortOrders: EnumSet<SortOrder> = EnumSet.of(
SortOrder.UPDATED,
SortOrder.ALPHABETICAL,
SortOrder.NEWEST,
@@ -42,7 +43,7 @@ internal class MangaDexParser(override val context: MangaLoaderContext) : MangaP
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val domain = getDomain()
val domain = domain
val url = buildString {
append("https://api.")
append(domain)
@@ -75,7 +76,7 @@ internal class MangaDexParser(override val context: MangaLoaderContext) : MangaP
},
)
}
val json = context.httpGet(url).parseJson().getJSONArray("data")
val json = webClient.httpGet(url).parseJson().getJSONArray("data")
return json.mapJSON { jo ->
val id = jo.getString("id")
val attrs = jo.getJSONObject("attributes")
@@ -123,70 +124,43 @@ internal class MangaDexParser(override val context: MangaLoaderContext) : MangaP
}
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val domain = getDomain()
val domain = domain
val mangaId = manga.url.removePrefix("/")
val attrsDeferred = async {
context.httpGet(
webClient.httpGet(
"https://api.$domain/manga/${mangaId}?includes[]=artist&includes[]=author&includes[]=cover_art",
).parseJson().getJSONObject("data").getJSONObject("attributes")
}
val feedDeferred = async { loadChapters(mangaId) }
val mangaAttrs = attrsDeferred.await()
val feed = feedDeferred.await()
// 2022-01-02T00:27:11+00:00
val dateFormat = SimpleDateFormat(
"yyyy-MM-dd'T'HH:mm:ss'+00:00'",
Locale.ROOT,
)
manga.copy(
description = mangaAttrs.getJSONObject("description").selectByLocale()
description = mangaAttrs.optJSONObject("description")?.selectByLocale()
?: manga.description,
chapters = feed.mapChapters { _, jo ->
val id = jo.getString("id")
val attrs = jo.getJSONObject("attributes")
if (!attrs.isNull("externalUrl")) {
return@mapChapters null
}
val locale = attrs.getStringOrNull("translatedLanguage")?.let { Locale.forLanguageTag(it) }
val relations = jo.getJSONArray("relationships").associateByKey("type")
val number = attrs.getIntOrDefault("chapter", 0)
MangaChapter(
id = generateUid(id),
name = attrs.getStringOrNull("title")?.takeUnless(String::isEmpty)
?: "Chapter #$number",
number = number,
url = id,
scanlator = relations["scanlation_group"]?.getStringOrNull("name"),
uploadDate = dateFormat.tryParse(attrs.getString("publishAt")),
branch = locale?.getDisplayName(locale)?.toTitleCase(locale),
source = source,
)
},
chapters = mapChapters(feed),
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val domain = getDomain()
val chapterJson = context.httpGet("https://api.$domain/at-home/server/${chapter.url}?forcePort443=false")
val domain = domain
val chapterJson = webClient.httpGet("https://api.$domain/at-home/server/${chapter.url}?forcePort443=false")
.parseJson()
.getJSONObject("chapter")
val pages = chapterJson.getJSONArray("data")
val prefix = "https://uploads.$domain/data/${chapterJson.getString("hash")}/"
val referer = "https://$domain/"
return List(pages.length()) { i ->
val url = prefix + pages.getString(i)
MangaPage(
id = generateUid(url),
url = url,
referer = referer,
preview = null, // TODO prefix + dataSaver.getString(i),
source = source,
)
}
}
override suspend fun getTags(): Set<MangaTag> {
val tags = context.httpGet("https://api.${getDomain()}/manga/tag").parseJson()
override suspend fun getAvailableTags(): Set<MangaTag> {
val tags = webClient.httpGet("https://api.${domain}/manga/tag").parseJson()
.getJSONArray("data")
return tags.mapJSONToSet { jo ->
MangaTag(
@@ -214,7 +188,7 @@ internal class MangaDexParser(override val context: MangaLoaderContext) : MangaP
return firstPage.data
}
val tail = coroutineScope {
val leftCount = firstPage.total - firstPage.size
val leftCount = firstPage.total.coerceAtMost(CHAPTERS_MAX_COUNT) - firstPage.size
val pages = (leftCount / CHAPTERS_MAX_PAGE_SIZE.toFloat()).toIntUp()
val dispatcher = Dispatchers.Default.limitedParallelism(CHAPTERS_PARALLELISM)
List(pages) { page ->
@@ -231,20 +205,25 @@ internal class MangaDexParser(override val context: MangaLoaderContext) : MangaP
}
private suspend fun loadChapters(mangaId: String, offset: Int, limit: Int): Chapters {
val limitedLimit = when {
offset >= CHAPTERS_MAX_COUNT -> return Chapters(emptyList(), CHAPTERS_MAX_COUNT)
offset + limit > CHAPTERS_MAX_COUNT -> CHAPTERS_MAX_COUNT - offset
else -> limit
}
val url = buildString {
append("https://api.")
append(getDomain())
append(domain)
append("/manga/")
append(mangaId)
append("/feed")
append("?limit=")
append(limit)
append(limitedLimit)
append("&includes[]=scanlation_group&order[volume]=asc&order[chapter]=asc&offset=")
append(offset)
append('&')
append(CONTENT_RATING)
}
val json = context.httpGet(url).parseJson()
val json = webClient.httpGet(url).parseJson()
if (json.getString("result") == "ok") {
return Chapters(
data = json.optJSONArray("data")?.toJSONList().orEmpty(),
@@ -258,6 +237,48 @@ internal class MangaDexParser(override val context: MangaLoaderContext) : MangaP
}
}
private fun mapChapters(list: List<JSONObject>): List<MangaChapter> {
// 2022-01-02T00:27:11+00:00
val dateFormat = SimpleDateFormat(
"yyyy-MM-dd'T'HH:mm:ss'+00:00'",
Locale.ROOT,
)
val chaptersBuilder = ChaptersListBuilder(list.size)
val branchedChapters = HashMap<String?, HashMap<Float, MangaChapter>>()
for (jo in list) {
val id = jo.getString("id")
val attrs = jo.getJSONObject("attributes")
if (!attrs.isNull("externalUrl")) {
continue
}
val number = jo.getJSONObject("attributes").getFloatOrDefault("chapter", 0f)
val locale = attrs.getStringOrNull("translatedLanguage")?.let { Locale.forLanguageTag(it) }
val lc = locale?.getDisplayName(locale)?.toTitleCase(locale)
val relations = jo.getJSONArray("relationships").associateByKey("type")
val team = relations["scanlation_group"]?.getJSONObject("attributes")?.getStringOrNull("name")
?.takeUnless { it.isBlank() }
val branch = (list.indices).firstNotNullOf { i ->
val b = if (i == 0) lc else "$lc ($i)"
if (branchedChapters[b]?.get(number) == null) b else null
}
val chapter = MangaChapter(
id = generateUid(id),
name = attrs.getStringOrNull("title")?.takeUnless(String::isEmpty)
?: "Chapter #${number.toString().removeSuffix(".0")}",
number = if (number <= 0f) (branchedChapters[branch]?.size?.plus(1) ?: 0) else number.toInt(),
url = id,
scanlator = team,
uploadDate = dateFormat.tryParse(attrs.getString("publishAt")),
branch = branch,
source = source,
)
if (chaptersBuilder.add(chapter)) {
branchedChapters.getOrPut(branch, ::HashMap)[number] = chapter
}
}
return chaptersBuilder.toList()
}
private class Chapters(
val data: List<JSONObject>,
val total: Int,
@@ -266,4 +287,4 @@ internal class MangaDexParser(override val context: MangaLoaderContext) : MangaP
val size: Int
get() = data.size
}
}
}

View File

@@ -1,7 +1,11 @@
package org.koitharu.kotatsu.parsers.site
package org.koitharu.kotatsu.parsers.site.all
import okhttp3.Headers
import androidx.collection.ArrayMap
import kotlinx.coroutines.sync.Mutex
import kotlinx.coroutines.sync.withLock
import okhttp3.HttpUrl.Companion.toHttpUrl
import okhttp3.Interceptor
import okhttp3.Response
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
@@ -12,25 +16,35 @@ import java.text.SimpleDateFormat
import java.util.*
internal abstract class NineMangaParser(
final override val context: MangaLoaderContext,
context: MangaLoaderContext,
source: MangaSource,
defaultDomain: String,
) : PagedMangaParser(source, pageSize = 26) {
) : PagedMangaParser(context, source, pageSize = 26), Interceptor {
override val configKeyDomain = ConfigKey.Domain(defaultDomain, null)
override val configKeyDomain = ConfigKey.Domain(defaultDomain)
init {
context.cookieJar.insertCookies(getDomain(), "ninemanga_template_desk=yes")
context.cookieJar.insertCookies(domain, "ninemanga_template_desk=yes")
}
override val headers = Headers.Builder()
override val headers = super.headers.newBuilder()
.add("Accept-Language", "en-US;q=0.7,en;q=0.3")
.build()
override val sortOrders: Set<SortOrder> = Collections.singleton(
override val availableSortOrders: Set<SortOrder> = Collections.singleton(
SortOrder.POPULARITY,
)
override fun intercept(chain: Interceptor.Chain): Response {
val request = chain.request()
val newRequest = if (request.url.host == domain) {
request.newBuilder().removeHeader("Referer").build()
} else {
request
}
return chain.proceed(newRequest)
}
override suspend fun getListPage(
page: Int,
query: String?,
@@ -39,7 +53,7 @@ internal abstract class NineMangaParser(
): List<Manga> {
val url = buildString {
append("https://")
append(getDomain())
append(domain)
when {
!query.isNullOrEmpty() -> {
append("/search/?name_sel=&wd=")
@@ -63,13 +77,11 @@ internal abstract class NineMangaParser(
append(page)
append(".html")
}
val doc = context.httpGet(url, headers).parseHtml()
val root = doc.body().selectFirst("ul.direlist")
?: doc.parseFailed("Cannot find root")
val doc = webClient.httpGet(url).parseHtml()
val root = doc.body().selectFirst("ul.direlist") ?: doc.parseFailed("Cannot find root")
val baseHost = root.baseUri().toHttpUrl().host
return root.select("li").map { node ->
val href = node.selectFirst("a")?.absUrl("href")
?: node.parseFailed("Link not found")
val href = node.selectFirst("a")?.absUrl("href") ?: node.parseFailed("Link not found")
val relUrl = href.toRelativeUrl(baseHost)
val dd = node.selectFirst("dd")
Manga(
@@ -91,30 +103,25 @@ internal abstract class NineMangaParser(
}
override suspend fun getDetails(manga: Manga): Manga {
val doc = context.httpGet(
manga.url.toAbsoluteUrl(getDomain()) + "?waring=1",
headers,
val doc = webClient.httpGet(
manga.url.toAbsoluteUrl(domain) + "?waring=1",
).parseHtml()
val root = doc.body().selectFirstOrThrow("div.manga")
val infoRoot = root.selectFirstOrThrow("div.bookintro")
val tagMap = getOrCreateTagMap()
val selectTag = infoRoot.getElementsByAttributeValue("itemprop", "genre").first()?.select("a")
val tags = selectTag?.mapNotNullToSet { tagMap[it.text()] }
return manga.copy(
tags = infoRoot.getElementsByAttributeValue("itemprop", "genre").first()
?.select("a")?.mapToSet { a ->
MangaTag(
title = a.text().toTitleCase(),
key = a.attr("href").substringBetween("/", "."),
source = source,
)
}.orEmpty(),
tags = tags.orEmpty(),
author = infoRoot.getElementsByAttributeValue("itemprop", "author").first()?.text(),
state = parseStatus(infoRoot.select("li a.red").text()),
description = infoRoot.getElementsByAttributeValue("itemprop", "description").first()
?.html()?.substringAfter("</b>"),
description = infoRoot.getElementsByAttributeValue("itemprop", "description").first()?.html()
?.substringAfter("</b>"),
chapters = root.selectFirst("div.chapterbox")?.select("ul.sub_vol_ul > li")
?.asReversed()?.mapChapters { i, li ->
?.mapChapters(reversed = true) { i, li ->
val a = li.selectFirst("a.chapter_list_a")
val href = a?.attrAsRelativeUrlOrNull("href")
?.replace("%20", " ") ?: li.parseFailed("Link not found")
val href =
a?.attrAsRelativeUrlOrNull("href")?.replace("%20", " ") ?: li.parseFailed("Link not found")
MangaChapter(
id = generateUid(href),
name = a.text(),
@@ -130,13 +137,12 @@ internal abstract class NineMangaParser(
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val doc = context.httpGet(chapter.url.toAbsoluteUrl(getDomain()), headers).parseHtml()
val doc = webClient.httpGet(chapter.url.toAbsoluteUrl(domain)).parseHtml()
return doc.body().getElementById("page")?.select("option")?.map { option ->
val url = option.attr("value")
MangaPage(
id = generateUid(url),
url = url,
referer = chapter.url.toAbsoluteUrl(getDomain()),
preview = null,
source = source,
)
@@ -144,30 +150,58 @@ internal abstract class NineMangaParser(
}
override suspend fun getPageUrl(page: MangaPage): String {
val doc = context.httpGet(page.url.toAbsoluteUrl(getDomain()), headers).parseHtml()
val doc = webClient.httpGet(page.url.toAbsoluteUrl(domain)).parseHtml()
val root = doc.body()
return root.selectFirst("a.pic_download")?.absUrl("href")
?: doc.parseFailed("Page image not found")
return root.selectFirst("a.pic_download")?.absUrl("href") ?: doc.parseFailed("Page image not found")
}
override suspend fun getTags(): Set<MangaTag> {
val doc = context.httpGet("https://${getDomain()}/search/?type=high", headers)
.parseHtml()
val root = doc.body().getElementById("search_form")
return root?.select("li.cate_list")?.mapNotNullToSet { li ->
val cateId = li.attr("cate_id") ?: return@mapNotNullToSet null
val a = li.selectFirst("a") ?: return@mapNotNullToSet null
MangaTag(
private var tagCache: ArrayMap<String, MangaTag>? = null
private val mutex = Mutex()
override suspend fun getAvailableTags(): Set<MangaTag> {
return getOrCreateTagMap().values.toSet()
}
private suspend fun getOrCreateTagMap(): Map<String, MangaTag> = mutex.withLock {
tagCache?.let { return@withLock it }
val tagMap = ArrayMap<String, MangaTag>()
val tagElements = webClient.httpGet("https://${domain}/search/?type=high").parseHtml().select("li.cate_list")
for (el in tagElements) {
if (el.text().isEmpty()) continue
val cateId = el.attr("cate_id")
val a = el.selectFirstOrThrow("a")
tagMap[el.text()] = MangaTag(
title = a.text().toTitleCase(),
key = cateId,
source = source,
)
} ?: doc.parseFailed("Root not found")
}
tagCache = tagMap
return@withLock tagMap
}
private fun parseStatus(status: String) = when {
// en
status.contains("Ongoing") -> MangaState.ONGOING
status.contains("Completed") -> MangaState.FINISHED
//es
status.contains("En curso") -> MangaState.ONGOING
status.contains("Completado") -> MangaState.FINISHED
//ru
status.contains("постоянный") -> MangaState.ONGOING
status.contains("завершенный") -> MangaState.FINISHED
//de
status.contains("Laufende") -> MangaState.ONGOING
status.contains("Abgeschlossen") -> MangaState.FINISHED
//pt
status.contains("Completo") -> MangaState.ONGOING
status.contains("Em tradução") -> MangaState.FINISHED
//it
status.contains("In corso") -> MangaState.ONGOING
status.contains("Completato") -> MangaState.FINISHED
//fr
status.contains("En cours") -> MangaState.ONGOING
status.contains("Complété") -> MangaState.FINISHED
else -> null
}
@@ -257,4 +291,4 @@ internal abstract class NineMangaParser(
MangaSource.NINEMANGA_FR,
"fr.ninemanga.com",
)
}
}

View File

@@ -0,0 +1,201 @@
package org.koitharu.kotatsu.parsers.site.animebootstrap
import kotlinx.coroutines.async
import kotlinx.coroutines.coroutineScope
import org.json.JSONArray
import org.jsoup.nodes.Document
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.util.*
// see https://themewagon.com/themes/free-bootstrap-4-html5-gaming-anime-website-template-anime/
internal abstract class AnimeBootstrapParser(
context: MangaLoaderContext,
source: MangaSource,
domain: String,
pageSize: Int = 24,
) : PagedMangaParser(context, source, pageSize) {
override val configKeyDomain = ConfigKey.Domain(domain)
override val isMultipleTagsSupported = false
override val availableSortOrders: Set<SortOrder> = EnumSet.of(
SortOrder.UPDATED,
SortOrder.POPULARITY,
SortOrder.ALPHABETICAL,
SortOrder.NEWEST,
)
protected open val listUrl = "/manga"
protected open val datePattern = "dd MMM. yyyy"
init {
paginator.firstPage = 1
searchPaginator.firstPage = 1
}
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val tag = tags.oneOrThrowIfMany()
val url = buildString {
append("https://")
append(domain)
append(listUrl)
append("?page=")
append(page.toString())
append("&type=all")
if (!query.isNullOrEmpty()) {
append("&search=")
append(query.urlEncoded())
}
if (!tags.isNullOrEmpty()) {
append("&categorie=")
append(tag?.key.orEmpty())
}
append("&sort=")
when (sortOrder) {
SortOrder.POPULARITY -> append("view")
SortOrder.UPDATED -> append("updated")
SortOrder.ALPHABETICAL -> append("default")
SortOrder.NEWEST -> append("published")
else -> append("updated")
}
}
val doc = webClient.httpGet(url).parseHtml()
return doc.select("div.col-6 div.product__item").map { div ->
val href = div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(div.host ?: domain),
coverUrl = div.selectFirstOrThrow("div.product__item__pic").attr("data-setbg").orEmpty(),
title = div.selectFirstOrThrow("div.product__item__text").text().orEmpty(),
altTitle = null,
rating = RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = null,
source = source,
isNsfw = isNsfwSource,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
val doc = webClient.httpGet("https://$domain$listUrl").parseHtml()
return doc.select("div.product__page__filter div:contains(Genre:) option ").mapNotNullToSet { option ->
val key = option.attr("value") ?: return@mapNotNullToSet null
val name = option.text()
MangaTag(
key = key,
title = name,
source = source,
)
}
}
protected open val selectDesc = "div.anime__details__text p"
protected open val selectState = "div.anime__details__widget li:contains(Ongoing)"
protected open val selectTag = "div.anime__details__widget li:contains(Categorie) a"
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val fullUrl = manga.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
val chaptersDeferred = async { getChapters(doc) }
val desc = doc.selectFirstOrThrow(selectDesc).html()
val state = if (doc.select(selectState).isNullOrEmpty()) {
MangaState.FINISHED
} else {
MangaState.ONGOING
}
manga.copy(
tags = doc.body().select(selectTag).mapNotNullToSet { a ->
MangaTag(
key = a.attr("href").substringAfterLast('='),
title = a.text().toTitleCase().replace(",", ""),
source = source,
)
},
description = desc,
state = state,
chapters = chaptersDeferred.await(),
)
}
protected open val selectChapter = "div.anime__details__episodes a"
protected open suspend fun getChapters(doc: Document): List<MangaChapter> {
return doc.body().select(selectChapter).mapChapters(reversed = true) { i, a ->
val href = a.attr("href")
MangaChapter(
id = generateUid(href),
name = a.text(),
number = i + 1,
url = href,
uploadDate = 0,
source = source,
scanlator = null,
branch = null,
)
}
}
protected open val selectPage = "div.read-img img"
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
if (doc.select("script:containsData(page_image)").isNullOrEmpty()) {
return doc.select(selectPage).map { img ->
val url = img.attr("onerror").replace("this.onerror=null;this.src=`", "").replace("`;", "")
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
)
}
} else {
val script = doc.selectFirstOrThrow("script:containsData(page_image)")
val images = JSONArray(script.data().substringAfterLast("var pages = ").substringBefore(';'))
val pages = ArrayList<MangaPage>(images.length())
for (i in 0 until images.length()) {
val pageTake = images.getJSONObject(i)
pages.add(
MangaPage(
id = generateUid(pageTake.getString("page_image")),
url = pageTake.getString("page_image"),
preview = null,
source = source,
),
)
}
return pages
}
}
}

View File

@@ -0,0 +1,149 @@
package org.koitharu.kotatsu.parsers.site.animebootstrap.fr
import kotlinx.coroutines.async
import kotlinx.coroutines.coroutineScope
import org.jsoup.nodes.Document
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.site.animebootstrap.AnimeBootstrapParser
import org.koitharu.kotatsu.parsers.util.*
import java.text.SimpleDateFormat
import java.util.EnumSet
import java.util.Locale
@MangaSourceParser("PAPSCAN", "PapScan", "fr")
internal class PapScan(context: MangaLoaderContext) :
AnimeBootstrapParser(context, MangaSource.PAPSCAN, "papscan.com") {
override val sourceLocale: Locale = Locale.ENGLISH
override val isMultipleTagsSupported = false
override val listUrl = "/liste-manga"
override val selectState = "div.anime__details__widget li:contains(En cours)"
override val selectTag = "div.anime__details__widget li:contains(Genre) a"
override val selectChapter = "ul.chapters li"
override val availableSortOrders: Set<SortOrder> = EnumSet.of(
SortOrder.POPULARITY,
SortOrder.ALPHABETICAL,
)
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val tag = tags.oneOrThrowIfMany()
val url = buildString {
append("https://")
append(domain)
append("/filterList")
append("?page=")
append(page.toString())
if (!query.isNullOrEmpty()) {
append("&alpha=")
append(query.urlEncoded())
}
if (!tags.isNullOrEmpty()) {
append("&cat=")
append(tag?.key.orEmpty())
}
append("&sortBy=")
when (sortOrder) {
SortOrder.POPULARITY -> append("views")
SortOrder.ALPHABETICAL -> append("name")
else -> append("updated")
}
}
val doc = webClient.httpGet(url).parseHtml()
return doc.select("div.product__item").map { div ->
val href = div.selectFirstOrThrow("h5 a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(div.host ?: domain),
coverUrl = div.selectFirstOrThrow("div.product__item__pic").attr("data-setbg").orEmpty(),
title = div.selectFirstOrThrow("div.product__item__text h5").text().orEmpty(),
altTitle = null,
rating = RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = null,
source = source,
isNsfw = isNsfwSource,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
val doc = webClient.httpGet("https://$domain$listUrl").parseHtml()
return doc.select("a.category ").mapNotNullToSet { a ->
val key = a.attr("href").substringAfterLast('=')
val name = a.text()
MangaTag(
key = key,
title = name,
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val fullUrl = manga.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
val chaptersDeferred = async { getChapters(doc) }
val desc = doc.selectFirstOrThrow(selectDesc).html()
val state = if (doc.select(selectState).isNullOrEmpty()) {
MangaState.FINISHED
} else {
MangaState.ONGOING
}
manga.copy(
tags = doc.body().select(selectTag).mapNotNullToSet { a ->
MangaTag(
key = a.attr("href").removeSuffix('/').substringAfterLast('/'),
title = a.text().toTitleCase(),
source = source,
)
},
description = desc,
state = state,
chapters = chaptersDeferred.await(),
)
}
override suspend fun getChapters(doc: Document): List<MangaChapter> {
val dateFormat = SimpleDateFormat(datePattern, sourceLocale)
return doc.body().select(selectChapter).mapChapters(reversed = true) { i, li ->
val href = li.selectFirstOrThrow("a").attr("href")
val dateText = li.selectFirst("span.date-chapter-title-rtl")?.text()
MangaChapter(
id = generateUid(href),
name = li.selectFirstOrThrow("span em").text(),
number = i + 1,
url = href,
uploadDate = dateFormat.tryParse(dateText),
source = source,
scanlator = null,
branch = null,
)
}
}
}

View File

@@ -0,0 +1,10 @@
package org.koitharu.kotatsu.parsers.site.animebootstrap.id
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.site.animebootstrap.AnimeBootstrapParser
@MangaSourceParser("KOMIKZOID", "Komikzo Id", "id")
internal class KomikzoId(context: MangaLoaderContext) :
AnimeBootstrapParser(context, MangaSource.KOMIKZOID, "komikzoid.xyz")

View File

@@ -0,0 +1,12 @@
package org.koitharu.kotatsu.parsers.site.animebootstrap.id
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.site.animebootstrap.AnimeBootstrapParser
@MangaSourceParser("NEUMANGA", "NeuManga", "id")
internal class NeuManga(context: MangaLoaderContext) :
AnimeBootstrapParser(context, MangaSource.NEUMANGA, "neumanga.xyz")

View File

@@ -0,0 +1,11 @@
package org.koitharu.kotatsu.parsers.site.animebootstrap.id
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.site.animebootstrap.AnimeBootstrapParser
@MangaSourceParser("SEKTEKOMIK", "SekteKomik", "id")
internal class SekteKomik(context: MangaLoaderContext) :
AnimeBootstrapParser(context, MangaSource.SEKTEKOMIK, "sektekomik.xyz")

View File

@@ -0,0 +1,167 @@
package org.koitharu.kotatsu.parsers.site.ar
import kotlinx.coroutines.async
import kotlinx.coroutines.coroutineScope
import org.json.JSONArray
import org.json.JSONObject
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import org.koitharu.kotatsu.parsers.util.json.mapJSON
import org.koitharu.kotatsu.parsers.util.json.mapJSONIndexed
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("FLIXSCANS", "FlixScans", "ar")
internal class FlixScans(context: MangaLoaderContext) : PagedMangaParser(context, MangaSource.FLIXSCANS, 18) {
override val availableSortOrders: Set<SortOrder> = EnumSet.of(SortOrder.UPDATED)
override val configKeyDomain = ConfigKey.Domain("flixscans.com")
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val json = if (!query.isNullOrEmpty()) {
if (page > 1) {
return emptyList()
}
val url = "https://api.$domain/api/v1/search/serie"
val body = JSONObject()
body.put("title", query.urlEncoded())
webClient.httpPost(url, body).parseJson().getJSONArray("data")
} else if (!tags.isNullOrEmpty()) {
if (page > 1) {
return emptyList()
}
val tagQuery = tags.joinToString(separator = ",") { it.key }
val url = "https://api.$domain/api/v1/search/advance?=&genres=$tagQuery&serie_type=webtoon"
webClient.httpGet(url).parseJson().getJSONArray("data")
} else {
val url = "https://api.$domain/api/v1/webtoon/homepage/latest/home?page=$page"
webClient.httpGet(url).parseJson().getJSONArray("data")
}
return json.mapJSON { j ->
val href = "https://$domain/series/${j.getString("prefix")}-${j.getString("id")}-${j.getString("slug")}"
val cover = "https://api.$domain/storage/" + j.getString("thumbnail")
Manga(
id = generateUid(href),
title = j.getString("title"),
altTitle = null,
url = href,
publicUrl = href.toAbsoluteUrl(domain),
rating = RATING_UNKNOWN,
isNsfw = false,
coverUrl = cover,
tags = emptySet(),
state = when (j.getString("status")) {
"ongoing" -> MangaState.ONGOING
"completed" -> MangaState.FINISHED
else -> null
},
author = null,
source = source,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
val doc = webClient.httpGet("https://$domain/search/advance").parseHtml()
val json = JSONArray(doc.requireElementById("__NUXT_DATA__").data())
val tagsList = json.getJSONArray(3).toString().replace("[", "").replace("]", "").split(",")
return tagsList.mapNotNullToSet { idTag ->
val id = idTag.toInt()
val idKey = json.getJSONObject(id).getInt("id")
val key = json.get(idKey).toString()
val idName = json.getJSONObject(id).getInt("name")
val name = json.get(idName).toString()
MangaTag(
key = key,
title = name,
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val doc = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
val chaptersDeferred = async { loadChapters(manga.url) }
val json = JSONArray(doc.requireElementById("__NUXT_DATA__").data())
val descId = json.getJSONObject(6).getInt("story")
val desc = json.getString(descId)
val tagsId = json.getJSONObject(6).getInt("genres")
val tagsList = json.getJSONArray(tagsId).toString().replace("[", "").replace("]", "").split(",")
val ratingId = json.getJSONObject(6).getInt("rating")
val rating = json.getString(ratingId)
val nsfwId = json.getJSONObject(6).getInt("nsfw")
val nsfw = json.getBoolean(nsfwId)
manga.copy(
description = desc,
tags = tagsList.mapToSet { idTag ->
val id = idTag.toInt()
val idKey = json.getJSONObject(id).getInt("id")
val key = json.get(idKey).toString()
val idName = json.getJSONObject(id).getInt("name")
val name = json.get(idName).toString()
MangaTag(
key = key,
title = name,
source = source,
)
},
rating = rating?.toFloatOrNull()?.div(5f) ?: RATING_UNKNOWN,
isNsfw = nsfw,
chapters = chaptersDeferred.await(),
)
}
private val dateFormat = SimpleDateFormat("yyyy-MM-dd", sourceLocale)
private suspend fun loadChapters(baseUrl: String): List<MangaChapter> {
val key = baseUrl.substringAfter("-").substringBefore("-")
val seriesKey = baseUrl.substringAfterLast("/").substringBefore("-")
val json = JSONArray(webClient.httpGet("https://api.$domain/api/v1/webtoon/chapters/$key-desc").parseRaw())
return json.mapJSONIndexed { i, j ->
val url = "https://$domain/read/webtoon/$seriesKey-${j.getString("id")}-${j.getString("slug")}"
val date = j.getString("createdAt").substringBeforeLast("T")
MangaChapter(
id = generateUid(url),
url = url,
name = j.getString("slug").replace('-', ' '),
number = i + 1,
branch = null,
uploadDate = dateFormat.tryParse(date),
scanlator = null,
source = source,
)
}
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
val json = JSONArray(doc.requireElementById("__NUXT_DATA__").data())
val chapterData = json.getJSONObject(6).getInt("chapterData")
val pageLocate = json.getJSONObject(chapterData).getInt("webtoon")
val jsonPages = json.getJSONArray(pageLocate)
val pages = ArrayList<MangaPage>(jsonPages.length())
for (i in 0 until jsonPages.length()) {
val id = jsonPages.getInt(i)
val url = "https://api.$domain/storage/" + json.getString(id)
pages.add(
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
),
)
}
return pages
}
}

View File

@@ -0,0 +1,121 @@
package org.koitharu.kotatsu.parsers.site.ar
import okhttp3.Headers
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.network.UserAgents
import org.koitharu.kotatsu.parsers.util.*
import java.util.*
@MangaSourceParser("MANGASTORM", "MangaStorm", "ar")
internal class MangaStorm(context: MangaLoaderContext) : PagedMangaParser(context, MangaSource.MANGASTORM, 30) {
override val availableSortOrders: Set<SortOrder> = EnumSet.of(SortOrder.POPULARITY)
override val configKeyDomain = ConfigKey.Domain("mangastorm.org")
override val isMultipleTagsSupported = false
override val headers: Headers = Headers.Builder()
.add("User-Agent", UserAgents.CHROME_DESKTOP)
.build()
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val tag = tags.oneOrThrowIfMany()
val url =
if (!tags.isNullOrEmpty()) {
buildString {
append("https://")
append(domain)
append("/categories/")
append(tag?.key.orEmpty())
append("?page=")
append(page)
}
} else {
buildString {
append("https://")
append(domain)
append("/mangas?page=")
append(page)
if (!query.isNullOrEmpty()) {
append("&query=")
append(query.urlEncoded())
}
}
}
val doc = webClient.httpGet(url).parseHtml()
return doc.select("div.row div.col").map { div ->
val href = div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
title = div.select(".manga-ct-title").text(),
altTitle = null,
url = href,
publicUrl = href.toAbsoluteUrl(domain),
rating = RATING_UNKNOWN,
isNsfw = false,
coverUrl = div.selectFirstOrThrow("img").src().orEmpty(),
tags = emptySet(),
state = null,
author = null,
source = source,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> = emptySet()
override suspend fun getDetails(manga: Manga): Manga {
val doc = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
val root = doc.selectFirstOrThrow(".card-body .col-lg-9")
return manga.copy(
altTitle = null,
state = null,
tags = root.select(".flex-wrap a").mapNotNullToSet { a ->
MangaTag(
key = a.attr("href").substringAfterLast('/'),
title = a.text(),
source = source,
)
},
author = null,
description = root.selectFirstOrThrow(".card-text").text(),
chapters = doc.select(".card-body a.btn-fixed-width").mapChapters(reversed = true) { i, a ->
val url = a.attrAsRelativeUrl("href")
MangaChapter(
id = generateUid(url),
name = a.text(),
number = i + 1,
url = url,
scanlator = null,
uploadDate = 0,
branch = null,
source = source,
)
},
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml().requireElementById("content")
return doc.select("div.text-center .img-fluid").map { img ->
val url = img.src()?.toRelativeUrl(domain) ?: img.parseFailed("Image src not found")
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
)
}
}
}

View File

@@ -0,0 +1,182 @@
package org.koitharu.kotatsu.parsers.site.ar
import kotlinx.coroutines.async
import kotlinx.coroutines.awaitAll
import kotlinx.coroutines.coroutineScope
import org.jsoup.nodes.Element
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("TEAMXNOVEL", "TeamXNovel", "ar")
internal class TeamXNovel(context: MangaLoaderContext) : PagedMangaParser(context, MangaSource.TEAMXNOVEL, 10) {
override val availableSortOrders: Set<SortOrder> = EnumSet.of(SortOrder.UPDATED, SortOrder.POPULARITY)
override val configKeyDomain = ConfigKey.Domain("team11x11.com")
override val isMultipleTagsSupported = false
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val tag = tags.oneOrThrowIfMany()
val url = buildString {
append("https://")
append(domain)
if (!tags.isNullOrEmpty()) {
append("/series?genre=")
append(tag?.key.orEmpty())
if (page > 1) {
append("&page=")
append(page)
}
} else if (!query.isNullOrEmpty()) {
append("/series?search=")
append(query.urlEncoded())
if (page > 1) {
append("&page=")
append(page)
}
} else {
when (sortOrder) {
SortOrder.POPULARITY -> append("/series")
SortOrder.UPDATED -> append("/")
else -> append("/")
}
if (page > 1) {
append("?page=")
append(page)
}
}
}
val doc = webClient.httpGet(url).parseHtml()
return doc.select("div.listupd .bs .bsx").ifEmpty {
doc.select("div.post-body .box")
}.map { div ->
val href = div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
title = div.select(".tt, h3").text(),
altTitle = null,
url = href,
publicUrl = href.toAbsoluteUrl(domain),
rating = RATING_UNKNOWN,
isNsfw = false,
coverUrl = div.selectFirstOrThrow("img").src().orEmpty(),
tags = emptySet(),
state = when (div.selectFirst(".status")?.text()) {
"مستمرة" -> MangaState.ONGOING
"متوقف", "مكتمل" -> MangaState.FINISHED
else -> null
},
author = null,
source = source,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
val doc = webClient.httpGet("https://$domain/series").parseHtml()
return doc.requireElementById("select_genre").select("option").mapNotNullToSet {
MangaTag(
key = it.attr("value"),
title = it.text(),
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga {
val doc = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
val mangaUrl = manga.url.toAbsoluteUrl(domain)
val maxPageChapterSelect = doc.select(".pagination .page-item a")
var maxPageChapter = 1
if (!maxPageChapterSelect.isNullOrEmpty()) {
maxPageChapterSelect.map {
val i = it.attr("href").substringAfterLast("=").toInt()
if (i > maxPageChapter) {
maxPageChapter = i
}
}
}
return manga.copy(
altTitle = null,
state = when (doc.selectFirstOrThrow(".full-list-info:contains(الحالة:) a").text()) {
"مستمرة" -> MangaState.ONGOING
"متوقف", "مكتمل" -> MangaState.FINISHED
else -> null
},
tags = doc.select(".review-author-info a").mapNotNullToSet { a ->
MangaTag(
key = a.attr("href").substringAfterLast("="),
title = a.text(),
source = source,
)
},
author = null,
description = doc.selectFirstOrThrow(".review-content").text(),
chapters = run {
if (maxPageChapter == 1) {
parseChapters(doc)
} else {
coroutineScope {
val result = ArrayList(parseChapters(doc))
result.ensureCapacity(result.size * maxPageChapter)
(2..maxPageChapter).map { i ->
async {
loadChapters(mangaUrl, i)
}
}.awaitAll()
.flattenTo(result)
result
}
}
}.reversed(),
)
}
private suspend fun loadChapters(baseUrl: String, page: Int): List<MangaChapter> {
return parseChapters(webClient.httpGet("$baseUrl?page=$page").parseHtml().body())
}
private val dateFormat = SimpleDateFormat("yyyy-MM-dd HH:mm:ss", sourceLocale)
private fun parseChapters(root: Element): List<MangaChapter> {
return root.requireElementById("chapter-contact").select(".eplister ul li")
.map { li ->
val url = li.selectFirstOrThrow("a").attrAsRelativeUrl("href")
MangaChapter(
id = generateUid(url),
name = li.selectFirstOrThrow(".epl-title").text(),
number = url.substringAfterLast('/').toIntOrNull() ?: 0,
url = url,
scanlator = null,
uploadDate = dateFormat.tryParse(li.selectFirstOrThrow(".epl-date").text()),
branch = null,
source = source,
)
}
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
return doc.select(".image_list img").map { img ->
val url = img.src()?.toRelativeUrl(domain) ?: img.parseFailed("Image src not found")
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
)
}
}
}

View File

@@ -1,4 +1,4 @@
package org.koitharu.kotatsu.parsers.site
package org.koitharu.kotatsu.parsers.site.be
import androidx.collection.ArraySet
import org.json.JSONArray
@@ -8,25 +8,25 @@ import org.koitharu.kotatsu.parsers.MangaParser
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.domain
import org.koitharu.kotatsu.parsers.util.generateUid
import org.koitharu.kotatsu.parsers.util.getDomain
import org.koitharu.kotatsu.parsers.util.json.mapJSON
import org.koitharu.kotatsu.parsers.util.json.mapJSONIndexed
import org.koitharu.kotatsu.parsers.util.json.mapJSONNotNull
import org.koitharu.kotatsu.parsers.util.json.stringIterator
import org.koitharu.kotatsu.parsers.util.toAbsoluteUrl
import java.util.*
@MangaSourceParser("ANIBEL", "Anibel", "be")
internal class AnibelParser(override val context: MangaLoaderContext) : MangaParser(MangaSource.ANIBEL) {
internal class AnibelParser(context: MangaLoaderContext) : MangaParser(context, MangaSource.ANIBEL) {
override val configKeyDomain = ConfigKey.Domain("anibel.net", null)
override val configKeyDomain = ConfigKey.Domain("anibel.net")
override val sortOrders: Set<SortOrder> = EnumSet.of(
override val availableSortOrders: Set<SortOrder> = EnumSet.of(
SortOrder.NEWEST,
)
override fun getFaviconUrl(): String {
return "https://cdn.${getDomain()}/favicons/favicon.png"
}
override suspend fun getList(
offset: Int,
query: String?,
@@ -73,12 +73,12 @@ internal class AnibelParser(override val context: MangaLoaderContext) : MangaPar
title = title.getString("be"),
coverUrl = jo.getString("poster").removePrefix("/cdn")
.toAbsoluteUrl(getDomain("cdn")) + "?width=200&height=280",
altTitle = title.getString("alt").takeUnless(String::isEmpty),
altTitle = title.optJSONArray("alt")?.optString(0)?.takeUnless(String::isEmpty),
author = null,
isNsfw = false,
rating = jo.getDouble("rating").toFloat() / 10f,
url = href,
publicUrl = "https://${getDomain()}/$href",
publicUrl = "https://${domain}/$href",
tags = jo.getJSONArray("genres").mapToTags(),
state = when (jo.getString("status")) {
"ongoing" -> MangaState.ONGOING
@@ -111,20 +111,21 @@ internal class AnibelParser(override val context: MangaLoaderContext) : MangaPar
""".trimIndent(),
).getJSONObject("media")
val title = details.getJSONObject("title")
val poster = details.getString("poster").removePrefix("/cdn")
.toAbsoluteUrl(getDomain("cdn"))
val poster = details.getString("poster").removePrefix("/cdn").toAbsoluteUrl(getDomain("cdn"))
val chapters = apiCall(
"""
chapters(mediaId: "${details.getString("mediaId")}") {
id
chapter
released
chapters(mediaId: "${details.getString("mediaId")}", offset: 0, limit: 99999) {
docs {
id
chapter
released
}
}
""".trimIndent(),
).getJSONArray("chapters")
).getJSONObject("chapters").getJSONArray("docs")
return manga.copy(
title = title.getString("be"),
altTitle = title.getString("alt"),
altTitle = title.optJSONArray("alt")?.optString(0)?.takeUnless(String::isEmpty),
coverUrl = "$poster?width=200&height=280",
largeCoverUrl = poster,
description = details.getJSONObject("description").getString("be"),
@@ -165,19 +166,17 @@ internal class AnibelParser(override val context: MangaLoaderContext) : MangaPar
""".trimIndent(),
).getJSONObject("chapter")
val pages = chapterJson.getJSONArray("images")
val chapterUrl = "https://${getDomain()}/${chapter.url}"
return pages.mapJSONIndexed { i, jo ->
MangaPage(
id = generateUid("${chapter.url}/$i"),
url = jo.getString("large"),
referer = chapterUrl,
preview = jo.getString("thumbnail"),
source = source,
)
}
}
override suspend fun getTags(): Set<MangaTag> {
override suspend fun getAvailableTags(): Set<MangaTag> {
val json = apiCall(
"""
getFilters(mediaType: manga) {
@@ -192,23 +191,29 @@ internal class AnibelParser(override val context: MangaLoaderContext) : MangaPar
private suspend fun search(query: String): List<Manga> {
val json = apiCall(
"""
search(query: "$query", limit: 40) {
id
search(query: "$query", limit: 60) {
mediaId
title {
be
en
}
poster
url
type
status
slug
mediaType
genres
}
""".trimIndent(),
)
val array = json.getJSONArray("search")
return array.mapJSON { jo ->
val mediaId = jo.getString("id")
return array.mapJSONNotNull { jo ->
val type = jo.getString("mediaType").lowercase()
if (type != "manga") {
return@mapJSONNotNull null
}
val mediaId = jo.getString("mediaId")
val title = jo.getJSONObject("title")
val href = "${jo.getString("type").lowercase()}/${jo.getString("url")}"
val href = "$type/${jo.getString("slug")}"
Manga(
id = generateUid(mediaId),
title = title.getString("be"),
@@ -219,17 +224,20 @@ internal class AnibelParser(override val context: MangaLoaderContext) : MangaPar
isNsfw = false,
rating = RATING_UNKNOWN,
url = href,
publicUrl = "https://${getDomain()}/$href",
tags = emptySet(),
state = null,
publicUrl = "https://${domain}/$href",
tags = jo.getJSONArray("genres").mapToTags(),
state = when (jo.getString("status")) {
"ongoing" -> MangaState.ONGOING
"finished" -> MangaState.FINISHED
else -> null
},
source = source,
)
}
}
private suspend fun apiCall(request: String): JSONObject {
return context.graphQLQuery("https://api.${getDomain()}/graphql", request)
.getJSONObject("data")
return webClient.graphQLQuery("https://${domain}/graphql", request).getJSONObject("data")
}
private fun JSONArray.mapToTags(): Set<MangaTag> {
@@ -242,6 +250,7 @@ internal class AnibelParser(override val context: MangaLoaderContext) : MangaPar
c == '-' -> {
builder.setCharAt(i, ' ')
}
capitalize -> {
builder.setCharAt(i, c.uppercaseChar())
capitalize = false
@@ -263,4 +272,4 @@ internal class AnibelParser(override val context: MangaLoaderContext) : MangaPar
}
return result
}
}
}

View File

@@ -0,0 +1,138 @@
package org.koitharu.kotatsu.parsers.site.en
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("BEETOON", "BeeToon.net", "en")
internal class BeeToon(context: MangaLoaderContext) :
PagedMangaParser(context, MangaSource.BEETOON, pageSize = 30) {
override val availableSortOrders: Set<SortOrder> = EnumSet.of(SortOrder.UPDATED, SortOrder.POPULARITY)
override val configKeyDomain = ConfigKey.Domain("ww7.beetoon.net")
override val isMultipleTagsSupported = false
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val tag = tags.oneOrThrowIfMany()
val url = buildString {
append("https://")
append(domain)
when {
!query.isNullOrEmpty() -> {
if (page > 1) {
return emptyList()
}
append("/?s=")
append(query.urlEncoded())
}
!tags.isNullOrEmpty() -> {
append("/genre/")
append(tag?.key.orEmpty())
append("/page-")
append(page)
append("/")
}
else -> {
when (sortOrder) {
SortOrder.UPDATED -> append("/latest-update/")
SortOrder.POPULARITY -> append("/popular-manga/")
else -> append("/latest-update/")
}
append("page-")
append(page)
append("/")
}
}
}
val doc = webClient.httpGet(url).parseHtml()
return doc.select(".comics-grid .entry").map { div ->
val href = div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(domain),
coverUrl = div.selectFirst("img")?.src().orEmpty(),
title = div.selectFirst(".name")?.text().orEmpty(),
altTitle = null,
rating = div.selectFirst(".counter")?.text()?.toFloatOrNull()?.div(5f) ?: RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = when (div.selectLastOrThrow(".status span").text()) {
"Ongoing" -> MangaState.ONGOING
"Completed" -> MangaState.FINISHED
else -> null
},
source = source,
isNsfw = isNsfwSource,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
val doc = webClient.httpGet("https://$domain/").parseHtml()
return doc.requireElementById("menu-item-3").select("ul.sub-menu li a").mapNotNullToSet {
MangaTag(
key = it.attr("href").removeSuffix('/').substringAfterLast('/'),
title = it.text(),
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga {
val doc = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
return manga.copy(
description = doc.getElementById("desc")?.text().orEmpty(),
rating = doc.selectFirst(".counter")?.text()?.toFloatOrNull()?.div(5f) ?: RATING_UNKNOWN,
tags = doc.body().select(".info .genre a").mapNotNullToSet {
MangaTag(
key = it.attr("href").removeSuffix('/').substringAfterLast('/'),
title = it.text(),
source = source,
)
},
author = doc.selectFirst(".info .author a")?.text(),
chapters = doc.select(".items-chapters a").mapChapters(reversed = true) { i, a ->
val url = a.attrAsRelativeUrl("href").toAbsoluteUrl(domain)
MangaChapter(
id = generateUid(url),
name = a.selectFirstOrThrow(".chap").text(),
number = i + 1,
url = url,
scanlator = null,
uploadDate = SimpleDateFormat("MM/dd/yyyy HH:mm:ss", Locale.ENGLISH)
.tryParse(a.selectFirst(".chapter-date")?.attr("title") ?: "0"),
branch = null,
source = source,
)
},
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val doc = webClient.httpGet(chapter.url.toAbsoluteUrl(domain)).parseHtml()
return doc.select(".chapter-content-inner center img").map { img ->
val urlPage = img.src()?.toRelativeUrl(domain) ?: img.parseFailed("Image src not found")
MangaPage(
id = generateUid(urlPage),
url = urlPage,
preview = null,
source = source,
)
}
}
}

View File

@@ -0,0 +1,93 @@
package org.koitharu.kotatsu.parsers.site.en
import org.koitharu.kotatsu.parsers.InternalParsersApi
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaParser
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.util.*
@MangaSourceParser("CLONEMANGA", "CloneManga", "en")
internal class CloneMangaParser(context: MangaLoaderContext) : MangaParser(context, MangaSource.CLONEMANGA) {
override val availableSortOrders: Set<SortOrder> = Collections.singleton(
SortOrder.POPULARITY,
)
override val configKeyDomain = ConfigKey.Domain("manga.clone-army.org")
@InternalParsersApi
override suspend fun getList(offset: Int, query: String?, tags: Set<MangaTag>?, sortOrder: SortOrder): List<Manga> {
if (query != null || offset > 0) {
return emptyList()
}
val link = "https://$domain/viewer_landing.php"
val doc = webClient.httpGet(link).parseHtml()
val mangas = doc.getElementsByClass("comicPreviewContainer")
return mangas.mapNotNull { item ->
val background = item.selectFirstOrThrow(".comicPreview").styleValueOrNull("background")
val href = item.selectFirst("a")?.attrAsRelativeUrl("href") ?: return@mapNotNull null
val cover = background?.substring(background.indexOf("site/themes"), background.indexOf(")"))
Manga(
id = generateUid(href),
title = item.selectFirst("h3")?.text() ?: return@mapNotNull null,
coverUrl = "https://$domain/$cover",
altTitle = null,
author = "Dan Kim",
rating = RATING_UNKNOWN,
url = href,
isNsfw = false,
tags = emptySet(),
state = null,
publicUrl = href.toAbsoluteUrl(domain),
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga {
val doc = webClient.httpGet(manga.publicUrl).parseHtml()
val series = doc.location()
val numChapters = Regex(
pattern = "&page=(.*)&lang=",
).findAll(
input = doc.getElementsByTag("script")[3].toString(),
)
.elementAt(3).destructured.component1()
.toInt()
val chapters = ArrayList<MangaChapter>()
for (i in 0..numChapters) {
val chapter = MangaChapter(
id = generateUid("$series&page=$i"),
name = "Chapter ${i + 1}",
number = i + 1,
url = "$series&page=$i",
scanlator = null,
branch = null,
uploadDate = 0L,
source = source,
)
chapters.add(chapter)
}
return manga.copy(chapters = chapters)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val doc = webClient.httpGet(chapter.url.toAbsoluteUrl(domain)).parseHtml()
val imgUrl = doc.getElementsByClass("subsectionContainer")[0]
.selectFirst("img")
?.attrAsAbsoluteUrlOrNull("src") ?: doc.parseFailed("Something broken")
return listOf(
MangaPage(
id = generateUid(imgUrl),
url = imgUrl,
preview = null,
source = source,
),
)
}
override suspend fun getAvailableTags(): Set<MangaTag> = emptySet()
}

View File

@@ -0,0 +1,149 @@
package org.koitharu.kotatsu.parsers.site.en
import okhttp3.Headers
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.network.UserAgents
import org.koitharu.kotatsu.parsers.util.*
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("COMICEXTRA", "ComicExtra", "en")
internal class ComicExtra(context: MangaLoaderContext) : PagedMangaParser(context, MangaSource.COMICEXTRA, 25) {
override val availableSortOrders: Set<SortOrder> =
EnumSet.of(SortOrder.POPULARITY, SortOrder.UPDATED, SortOrder.NEWEST)
override val configKeyDomain = ConfigKey.Domain("comicextra.me")
override val isMultipleTagsSupported = false
override val headers: Headers = Headers.Builder()
.add("User-Agent", UserAgents.CHROME_DESKTOP)
.build()
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val tag = tags.oneOrThrowIfMany()
val url = buildString {
append("https://$domain/")
if (!tags.isNullOrEmpty()) {
append(tag?.key.orEmpty())
if (page > 1) {
append("/")
append(page)
}
} else if (!query.isNullOrEmpty()) {
append("comic-search?key=")
append(query.urlEncoded())
if (page > 1) {
append("&page=")
append(page)
}
} else {
when (sortOrder) {
SortOrder.POPULARITY -> append("popular-comic/")
SortOrder.UPDATED -> append("new-comic/")
SortOrder.NEWEST -> append("recent-comic/")
else -> append("new-comic/")
}
if (page > 1) {
append(page)
}
}
}
val doc = webClient.httpGet(url).parseHtml()
return doc.select("div.movie-list-index div.cartoon-box").map { div ->
val href = div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
title = div.selectFirstOrThrow("h3").text(),
altTitle = null,
url = href,
publicUrl = href.toAbsoluteUrl(domain),
rating = RATING_UNKNOWN,
isNsfw = false,
coverUrl = div.selectFirstOrThrow("img").attrAsAbsoluteUrl("src"),
tags = emptySet(),
state = when (div.selectFirstOrThrow(".detail:contains(Stasus: )").text()) {
"Stasus: Ongoing" -> MangaState.ONGOING
"Stasus: Completed" -> MangaState.FINISHED
else -> null
},
author = null,
source = source,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
val doc = webClient.httpGet("https://$domain/popular-comic").parseHtml()
return doc.select("li.tag-item a").mapNotNullToSet { a ->
MangaTag(
key = a.attr("href").substringAfterLast('/'),
title = a.text(),
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga {
val doc = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
val dateFormat = SimpleDateFormat("MM/dd/yy", sourceLocale)
return manga.copy(
altTitle = doc.selectFirstOrThrow("dt.movie-dt:contains(Alternate name:) + dd").text(),
state = when (doc.selectFirstOrThrow("dt.movie-dt:contains(Status:) + dd a").text()) {
"Ongoing" -> MangaState.ONGOING
"Completed" -> MangaState.FINISHED
else -> null
},
tags = doc.select("dt.movie-dt:contains(Genres:) + dd a").mapNotNullToSet { a ->
MangaTag(
key = a.attr("href").substringAfterLast("/"),
title = a.text(),
source = source,
)
},
author = doc.select("dt.movie-dt:contains(Author:) + dd").text(),
description = doc.getElementById("film-content")?.text(),
chapters = doc.requireElementById("list").select("tr")
.mapChapters(reversed = true) { i, tr ->
val a = tr.selectFirstOrThrow("a")
val url = a.attrAsRelativeUrl("href") + "/full"
val name = a.text()
val dateText = tr.select("td").last()?.text()
MangaChapter(
id = generateUid(url),
name = name,
number = i + 1,
url = url,
scanlator = null,
uploadDate = dateFormat.tryParse(dateText),
branch = null,
source = source,
)
},
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
return doc.select(".chapter-container img").map { img ->
val url = img.src()?.toRelativeUrl(domain) ?: img.parseFailed("Image src not found")
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
)
}
}
}

View File

@@ -0,0 +1,170 @@
package org.koitharu.kotatsu.parsers.site.en
import okhttp3.Headers
import org.json.JSONArray
import org.jsoup.nodes.Document
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.network.UserAgents
import org.koitharu.kotatsu.parsers.util.*
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("DYNASTYSCANS", "DynastyScans", "en")
internal class DynastyScans(context: MangaLoaderContext) : PagedMangaParser(context, MangaSource.DYNASTYSCANS, 117) {
override val availableSortOrders: Set<SortOrder> = EnumSet.of(SortOrder.ALPHABETICAL)
override val configKeyDomain = ConfigKey.Domain("dynasty-scans.com")
override val headers: Headers = Headers.Builder()
.add("User-Agent", UserAgents.CHROME_DESKTOP)
.build()
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val url = buildString {
append("https://")
append(domain)
if (!query.isNullOrEmpty()) {
append("/search?q=")
append(query.urlEncoded())
append("&")
append("classes[]".urlEncoded())
append("=Serie&page=")
append(page.toString())
} else if (!tags.isNullOrEmpty()) {
append("/tags/")
for (tag in tags) {
append(tag.key)
}
append("?view=groupings&page=")
append(page.toString())
} else {
append("/series?view=cover&page=")
append(page.toString())
}
}
val doc = webClient.httpGet(url).parseHtml()
// There are no images on the search page
if (!query.isNullOrEmpty()) {
return doc.select("dl.chapter-list dd")
.map { div ->
val href = div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
title = div.selectFirstOrThrow("a").text(),
altTitle = null,
url = href,
publicUrl = href.toAbsoluteUrl(domain),
rating = RATING_UNKNOWN,
isNsfw = false,
coverUrl = "",
tags = div.select("span.tags a").mapNotNullToSet { a ->
MangaTag(
key = a.attr("href").removeSuffix('/').substringAfterLast('/'),
title = a.text(),
source = source,
)
},
state = null,
author = null,
source = source,
)
}
} else {
return doc.select("li.span2")
.map { div ->
val href = div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
title = div.selectFirstOrThrow("div.caption").text(),
altTitle = null,
url = href,
publicUrl = href.toAbsoluteUrl(domain),
rating = RATING_UNKNOWN,
isNsfw = false,
coverUrl = div.selectFirstOrThrow("img").attrAsAbsoluteUrl("src"),
tags = setOf(),
state = null,
author = null,
source = source,
)
}
}
}
override suspend fun getAvailableTags(): Set<MangaTag> = emptySet()
override suspend fun getDetails(manga: Manga): Manga {
val doc = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
val chapters = getChapters(doc)
val root = doc.requireElementById("main")
return manga.copy(
altTitle = null,
state = when (root.select("h2.tag-title small").last()?.text()) {
"— Ongoing" -> MangaState.ONGOING
"— Completed" -> MangaState.FINISHED
else -> null
},
coverUrl = root.selectFirst("img.thumbnail")?.src()
.orEmpty(), // It is needed if the manga was found via the search.
tags = root.select("div.tag-tags a").mapNotNullToSet { a ->
val href = a.attr("href").removeSuffix('/').substringAfterLast('/')
MangaTag(
key = href,
title = a.text(),
source = source,
)
},
author = null,
description = null,
chapters = chapters,
)
}
private fun getChapters(doc: Document): List<MangaChapter> {
val dateFormat = SimpleDateFormat("MMM dd yy", sourceLocale)
return doc.body().select("dl.chapter-list dd").mapChapters { i, li ->
val a = li.selectFirstOrThrow("a")
val href = a.attrAsRelativeUrl("href")
val dateText = li.select("small").last()?.text()?.replace("released ", "")?.replace("'", "")
MangaChapter(
id = generateUid(href),
name = a.text(),
number = i + 1,
url = href,
uploadDate = dateFormat.tryParse(dateText),
source = source,
scanlator = null,
branch = null,
)
}
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val chapterUrl = chapter.url.toAbsoluteUrl(domain)
val docs = webClient.httpGet(chapterUrl).parseHtml()
val script = docs.selectFirstOrThrow("script:containsData(var pages =)")
val json = JSONArray(script.data().substringAfter('=').substringBeforeLast(';'))
val pages = ArrayList<MangaPage>(json.length())
for (i in 0 until json.length()) {
val url = "https://" + domain + json.getString(i).substringAfter(":\"").substringBefore("\",")
pages.add(
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
),
)
}
return pages
}
}

View File

@@ -0,0 +1,148 @@
package org.koitharu.kotatsu.parsers.site.en
import kotlinx.coroutines.async
import kotlinx.coroutines.coroutineScope
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import org.koitharu.kotatsu.parsers.util.json.mapJSONToSet
import java.util.*
@MangaSourceParser("FAKKU", "Fakku", "en", ContentType.HENTAI)
internal class Fakku(context: MangaLoaderContext) :
PagedMangaParser(context, MangaSource.FAKKU, pageSize = 25) {
override val availableSortOrders: Set<SortOrder> =
EnumSet.of(SortOrder.ALPHABETICAL, SortOrder.NEWEST, SortOrder.UPDATED)
override val configKeyDomain = ConfigKey.Domain("fakku.cc")
override val isMultipleTagsSupported = false
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val tag = tags.oneOrThrowIfMany()
val url = buildString {
append("https://")
append(domain)
when {
!query.isNullOrEmpty() -> {
append("/search?q=")
append(query.urlEncoded())
append("&")
}
!tags.isNullOrEmpty() -> {
append("/tags/")
append(tag?.key.orEmpty())
append("?")
}
else -> {
append("?")
}
}
append("page=")
append(page)
append("&sort=")
when (sortOrder) {
SortOrder.ALPHABETICAL -> append("title")
SortOrder.NEWEST -> append("created_at")
SortOrder.UPDATED -> append("published_at")
else -> append("published_at")
}
}
val doc = webClient.httpGet(url).parseHtml()
return doc.select("div.entries .entry a").map { a ->
val href = a.attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(domain),
coverUrl = a.selectFirst("img")?.src().orEmpty(),
title = a.selectFirst(".title")?.text().orEmpty(),
altTitle = null,
rating = RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = null,
source = source,
isNsfw = isNsfwSource,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
val root = webClient.httpGet("https://$domain/tags").parseHtml()
return root.select("div.entries .entry a").mapToSet {
MangaTag(
key = it.attr("href").substringAfterLast("/"),
title = it.selectFirstOrThrow(".name").text(),
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val fullUrl = manga.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
val genreDeferred = async {
webClient.httpGet(manga.url.toAbsoluteUrl(domain) + ".json").parseJson()
}
val genre = genreDeferred.await()
manga.copy(
author = doc.selectFirst("tr.artists a")?.text(),
tags = if (genre.toString().contains("tags")) {
genre.getJSONArray("tags").mapJSONToSet {
MangaTag(
key = it.getString("slug"),
title = it.getString("name"),
source = source,
)
}
} else {
emptySet()
},
chapters = listOf(
MangaChapter(
id = manga.id,
name = manga.title,
number = 1,
url = manga.url + "/1",
scanlator = null,
uploadDate = 0,
branch = null,
source = source,
),
),
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val doc = webClient.httpGet(chapter.url.toAbsoluteUrl(domain)).parseHtml()
val totalPages = doc.selectFirstOrThrow(".total").text().toInt()
val rawUrl = chapter.url.substringBeforeLast("/")
return (1..totalPages).map {
val url = "$rawUrl/$it"
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
)
}
}
override suspend fun getPageUrl(page: MangaPage): String {
val doc = webClient.httpGet(page.url.toAbsoluteUrl(domain)).parseHtml()
val root = doc.body()
return root.selectFirstOrThrow(".page img").attrAsAbsoluteUrl("src")
}
}

View File

@@ -0,0 +1,187 @@
package org.koitharu.kotatsu.parsers.site.en
import androidx.collection.ArraySet
import kotlinx.coroutines.async
import kotlinx.coroutines.awaitAll
import kotlinx.coroutines.coroutineScope
import okhttp3.HttpUrl.Companion.toHttpUrlOrNull
import org.json.JSONArray
import org.jsoup.nodes.Element
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import org.koitharu.kotatsu.parsers.util.json.mapJSON
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("KSKMOE", "Ksk.moe", "en", ContentType.HENTAI)
internal class KskMoe(context: MangaLoaderContext) : PagedMangaParser(context, MangaSource.KSKMOE, 35) {
override val availableSortOrders: Set<SortOrder> =
EnumSet.of(SortOrder.UPDATED, SortOrder.POPULARITY, SortOrder.NEWEST, SortOrder.ALPHABETICAL)
override val configKeyDomain = ConfigKey.Domain("ksk.moe")
override val isMultipleTagsSupported = false
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val tag = tags.oneOrThrowIfMany()
val url = buildString {
append("https://")
append(domain)
if (!tags.isNullOrEmpty()) {
append("/tags/")
append(tag?.key.orEmpty())
} else {
append("/browse")
}
if (page > 1) {
append("/page/")
append(page)
}
when (sortOrder) {
SortOrder.POPULARITY -> append("?sort=32")
SortOrder.UPDATED -> append("")
SortOrder.NEWEST -> append("?sort=16")
SortOrder.ALPHABETICAL -> append("?sort=1")
else -> append("")
}
if (!query.isNullOrEmpty()) {
append("?s=")
append(query.urlEncoded())
}
}
val doc = webClient.httpGet(url).parseHtml()
if (!doc.html().contains("pagination") && page > 1) {
return emptyList()
}
return doc.requireElementById("galleries").select("article").map { div ->
val href = div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
title = div.selectLastOrThrow("h3 span").text(),
altTitle = null,
url = href,
publicUrl = href.toAbsoluteUrl(domain),
rating = RATING_UNKNOWN,
isNsfw = true,
coverUrl = div.selectFirstOrThrow("img").src()?.toAbsoluteUrl(domain).orEmpty(),
tags = div.select("footer span").mapNotNullToSet { span ->
MangaTag(
key = span.text().urlEncoded(),
title = span.text(),
source = source,
)
},
state = null,
author = null,
source = source,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
return coroutineScope {
(1..2).map { page ->
async { getTags(page) }
}
}.awaitAll().flattenTo(ArraySet(360))
}
private suspend fun getTags(page: Int): Set<MangaTag> {
val url = if (page == 1) {
"https://$domain/tags"
} else {
"https://$domain/tags/page/$page"
}
val root = webClient.httpGet(url).parseHtml().body().getElementById("tags")
return root?.parseTags().orEmpty()
}
private fun Element.parseTags() = select("section.tags div a").mapToSet { a ->
MangaTag(
key = a.attr("href").substringAfterLast("/tags/"),
title = a.selectFirstOrThrow("span").text(),
source = source,
)
}
private val date = SimpleDateFormat("dd.MM.yyyy hh:mm 'UTC'", Locale.US)
override suspend fun getDetails(manga: Manga): Manga {
val doc = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
return manga.copy(
tags = doc.requireElementById("metadata").select("main div:contains(Tag) a").mapNotNullToSet { a ->
MangaTag(
key = a.attr("href").substringAfterLast("/tags/"),
title = a.selectFirstOrThrow("span").text(),
source = source,
)
},
author = doc.requireElementById("metadata").selectFirstOrThrow("main div:contains(Artist) a span").text(),
chapters = listOf(
MangaChapter(
id = generateUid(manga.id),
name = manga.title,
number = 1,
url = manga.url,
scanlator = null,
uploadDate = date.tryParse(doc.selectFirstOrThrow("time.updated").text()),
branch = null,
source = source,
),
),
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url
.replace("/view/", "/read/")
.let { "$it/1" }
.toAbsoluteUrl(domain)
val document = webClient.httpGet(fullUrl).parseHtml()
val id = fullUrl
.substringAfter("/read/")
.substringBeforeLast("/")
val cdnUrl = document.selectFirst("meta[itemprop=image]")
?.attr("content")
?.toHttpUrlOrNull()
?.host
.let { "https://" + (it ?: domain) }
val script = document.select("script:containsData(window.metadata)").html()
val rawJson = script
.substringAfter("original:")
.substringBefore("resampled:")
.substringBeforeLast(",")
return JSONArray(rawJson).mapJSON {
val fileName = it.getString("n")
val url = "$cdnUrl/original/$id/$fileName"
val preview = "$cdnUrl/t/$id/320/$fileName"
MangaPage(
id = generateUid(url),
url = url,
preview = preview,
source = source,
)
}
}
}

View File

@@ -0,0 +1,148 @@
package org.koitharu.kotatsu.parsers.site.en
import okhttp3.Headers
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.network.UserAgents
import org.koitharu.kotatsu.parsers.util.*
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("MANGAGEKO", "MangaGeko", "en")
internal class MangaGeko(context: MangaLoaderContext) : PagedMangaParser(context, MangaSource.MANGAGEKO, 30) {
override val availableSortOrders: Set<SortOrder> =
EnumSet.of(SortOrder.POPULARITY, SortOrder.UPDATED, SortOrder.NEWEST)
override val configKeyDomain = ConfigKey.Domain("www.mangageko.com")
override val isMultipleTagsSupported = false
override val headers: Headers = Headers.Builder()
.add("User-Agent", UserAgents.CHROME_DESKTOP)
.build()
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val tag = tags.oneOrThrowIfMany()
val url = if (!query.isNullOrEmpty()) {
if (page > 1) {
return emptyList()
}
buildString {
append("https://$domain/search/?search=")
append(query.urlEncoded())
}
} else {
buildString {
append("https://$domain/browse-comics/?results=")
append(page)
append("&filter=")
when (sortOrder) {
SortOrder.POPULARITY -> append("views")
SortOrder.UPDATED -> append("Updated")
SortOrder.NEWEST -> append("New")
else -> append("Updated")
}
if (!tags.isNullOrEmpty()) {
append("&genre=")
append(tag?.key.orEmpty())
}
}
}
val doc = webClient.httpGet(url).parseHtml()
return doc.select("li.novel-item").map { div ->
val href = div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
title = div.selectFirstOrThrow("h4").text(),
altTitle = null,
url = href,
publicUrl = href.toAbsoluteUrl(domain),
rating = RATING_UNKNOWN,
isNsfw = false,
coverUrl = div.selectFirstOrThrow("img").src().orEmpty(),
tags = emptySet(),
state = null,
author = div.selectFirstOrThrow("h6").text().removePrefix("Author(S): "),
source = source,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
val doc = webClient.httpGet("https://$domain/browse-comics/").parseHtml()
return doc.select("label.checkbox-inline").mapNotNullToSet { label ->
MangaTag(
key = label.selectFirstOrThrow("input").attr("value"),
title = label.text(),
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga {
val doc = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
val dateFormat = SimpleDateFormat("MMM dd, yyyy", sourceLocale)
return manga.copy(
altTitle = doc.selectFirstOrThrow(".alternative-title").text(),
state = when (doc.selectFirstOrThrow(".header-stats span:contains(Status) strong").text()) {
"Ongoing" -> MangaState.ONGOING
"Completed" -> MangaState.FINISHED
else -> null
},
tags = doc.select(".categories ul li a").mapNotNullToSet { a ->
MangaTag(
key = a.attr("href").substringAfterLast("="),
title = a.text(),
source = source,
)
},
author = doc.selectFirstOrThrow(".author").text(),
description = doc.selectFirstOrThrow(".description").html(),
chapters = doc.requireElementById("chapters").select("ul.chapter-list li")
.mapChapters(reversed = true) { i, li ->
val a = li.selectFirstOrThrow("a")
val url = a.attrAsRelativeUrl("href")
val name = li.selectFirstOrThrow(".chapter-title").text()
val dateText = li.select(".chapter-update").attr("datetime").substringBeforeLast(',')
.replace(".", "").replace("Sept", "Sep")
MangaChapter(
id = generateUid(url),
name = name,
number = i + 1,
url = url,
scanlator = null,
uploadDate = dateFormat.tryParse(dateText),
branch = null,
source = source,
)
},
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
return doc.requireElementById("chapter-reader").select("img").map { img ->
val url = img.src()?.toRelativeUrl(domain) ?: img.parseFailed("Image src not found")
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
)
}
}
}

View File

@@ -1,4 +1,4 @@
package org.koitharu.kotatsu.parsers.site
package org.koitharu.kotatsu.parsers.site.en
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaParser
@@ -11,11 +11,11 @@ import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("MANGATOWN", "MangaTown", "en")
internal class MangaTownParser(override val context: MangaLoaderContext) : MangaParser(MangaSource.MANGATOWN) {
internal class MangaTownParser(context: MangaLoaderContext) : MangaParser(context, MangaSource.MANGATOWN) {
override val configKeyDomain = ConfigKey.Domain("www.mangatown.com", null)
override val configKeyDomain = ConfigKey.Domain("www.mangatown.com")
override val sortOrders: Set<SortOrder> = EnumSet.of(
override val availableSortOrders: Set<SortOrder> = EnumSet.of(
SortOrder.ALPHABETICAL,
SortOrder.RATING,
SortOrder.POPULARITY,
@@ -42,18 +42,18 @@ internal class MangaTownParser(override val context: MangaLoaderContext) : Manga
if (offset != 0) {
return emptyList()
}
"/search?name=${query.urlEncoded()}".toAbsoluteUrl(getDomain())
"/search?name=${query.urlEncoded()}".toAbsoluteUrl(domain)
}
tags.isNullOrEmpty() -> "/directory/$page.htm$sortKey".toAbsoluteUrl(getDomain())
tags.size == 1 -> "/directory/${tags.first().key}/$page.htm$sortKey".toAbsoluteUrl(getDomain())
tags.isNullOrEmpty() -> "/directory/$page.htm$sortKey".toAbsoluteUrl(domain)
tags.size == 1 -> "/directory/${tags.first().key}/$page.htm$sortKey".toAbsoluteUrl(domain)
else -> tags.joinToString(
prefix = "/search?page=$page".toAbsoluteUrl(getDomain()),
prefix = "/search?page=$page".toAbsoluteUrl(domain),
) { tag ->
"&genres[${tag.key}]=1"
}
}
val doc = context.httpGet(url).parseHtml()
val doc = webClient.httpGet(url).parseHtml()
val root = doc.body().selectFirstOrThrow("ul.manga_pic_list")
return root.select("li").mapNotNull { li ->
val a = li.selectFirst("a.manga_cover")
@@ -87,13 +87,13 @@ internal class MangaTownParser(override val context: MangaLoaderContext) : Manga
}.orEmpty(),
url = href,
isNsfw = false,
publicUrl = href.toAbsoluteUrl(a.host ?: getDomain()),
publicUrl = href.toAbsoluteUrl(a.host ?: domain),
)
}
}
override suspend fun getDetails(manga: Manga): Manga {
val doc = context.httpGet(manga.url.toAbsoluteUrl(getDomain())).parseHtml()
val doc = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
val root = doc.body().selectFirstOrThrow("section.main")
.selectFirstOrThrow("div.article_content")
val info = root.selectFirst("div.detail_info")?.selectFirst("ul")
@@ -135,8 +135,8 @@ internal class MangaTownParser(override val context: MangaLoaderContext) : Manga
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(getDomain())
val doc = context.httpGet(fullUrl).parseHtml()
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
val root = doc.body().selectFirstOrThrow("div.page_select")
return root.selectFirstOrThrow("select").selectOrThrow("option").mapNotNull {
val href = it.attrAsRelativeUrlOrNull("value")
@@ -147,19 +147,18 @@ internal class MangaTownParser(override val context: MangaLoaderContext) : Manga
id = generateUid(href),
url = href,
preview = null,
referer = fullUrl,
source = MangaSource.MANGATOWN,
)
}
}
override suspend fun getPageUrl(page: MangaPage): String {
val doc = context.httpGet(page.url.toAbsoluteUrl(getDomain())).parseHtml()
val doc = webClient.httpGet(page.url.toAbsoluteUrl(domain)).parseHtml()
return doc.requireElementById("image").attrAsAbsoluteUrl("src")
}
override suspend fun getTags(): Set<MangaTag> {
val doc = context.httpGet("/directory/".toAbsoluteUrl(getDomain())).parseHtml()
override suspend fun getAvailableTags(): Set<MangaTag> {
val doc = webClient.httpGet("/directory/".toAbsoluteUrl(domain)).parseHtml()
val root = doc.body().selectFirst("aside.right")
?.getElementsContainingOwnText("Genres")
?.first()
@@ -188,7 +187,7 @@ internal class MangaTownParser(override val context: MangaLoaderContext) : Manga
}
private suspend fun bypassLicensedChapters(manga: Manga): List<MangaChapter> {
val doc = context.httpGet(manga.url.toAbsoluteUrl(getDomain("m"))).parseHtml()
val doc = webClient.httpGet(manga.url.toAbsoluteUrl(getDomain("m"))).parseHtml()
val list = doc.body().selectFirst("ul.detail-ch-list") ?: return emptyList()
val dateFormat = SimpleDateFormat("MMM dd,yyyy", Locale.US)
return list.select("li").asReversed().mapIndexedNotNull { i, li ->
@@ -214,4 +213,4 @@ internal class MangaTownParser(override val context: MangaLoaderContext) : Manga
}
private fun String.parseTagKey() = split('/').findLast { regexTag matches it }
}
}

View File

@@ -0,0 +1,193 @@
package org.koitharu.kotatsu.parsers.site.en
import kotlinx.coroutines.coroutineScope
import okhttp3.Headers
import org.jsoup.nodes.Document
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.network.UserAgents
import org.koitharu.kotatsu.parsers.util.*
import org.koitharu.kotatsu.parsers.util.json.mapJSON
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("MANGAOWL", "MangaOwl.to", "en")
internal class Mangaowl(context: MangaLoaderContext) :
PagedMangaParser(context, MangaSource.MANGAOWL, pageSize = 24) {
override val availableSortOrders: Set<SortOrder> = EnumSet.of(
SortOrder.POPULARITY,
SortOrder.NEWEST,
SortOrder.UPDATED,
SortOrder.RATING,
)
override val configKeyDomain = ConfigKey.Domain("mangaowl.to")
override val headers: Headers = Headers.Builder()
.add("User-Agent", UserAgents.CHROME_DESKTOP)
.build()
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val sort = when (sortOrder) {
SortOrder.POPULARITY -> "view_count"
SortOrder.UPDATED -> "-modified_at"
SortOrder.NEWEST -> "created_at"
SortOrder.RATING -> "rating"
else -> "modified_at"
}
val url = buildString {
append("https://")
append(domain)
when {
!query.isNullOrEmpty() -> {
append("/8-search")
append("?q=")
append(query.urlEncoded())
append("&page=")
append(page.toString())
}
!tags.isNullOrEmpty() -> {
append("/8-genres/")
for (tag in tags) {
append(tag.key)
}
append("?page=")
append(page.toString())
}
else -> {
append("/8-comics")
append("?page=")
append(page.toString())
append("&ordering=")
append(sort)
}
}
}
val doc = webClient.httpGet(url).parseHtml()
return doc.select("div.manga-item.column").map { div ->
val href = div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(div.host ?: domain),
coverUrl = div.selectFirst("img")?.src().orEmpty(),
title = div.selectFirst("a.one-line")?.text().orEmpty(),
altTitle = null,
rating = div.select("span").last()?.text()?.toFloatOrNull()?.div(5f) ?: RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = null,
source = source,
isNsfw = false,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
val doc = webClient.httpGet("https://$domain/8-genres").parseHtml()
return doc.select("div.genres-container span.genre-item a").mapNotNullToSet { a ->
val key = a.attr("href").substringAfterLast("/")
MangaTag(
key = key,
title = a.text(),
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val fullUrl = manga.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
manga.copy(
tags = doc.body().select("div.comic-attrs div.column.my-2:contains(Genres) a").mapNotNullToSet { a ->
MangaTag(
key = a.attr("href").removeSuffix("/").substringAfterLast('/'),
title = a.text().toTitleCase().replace(",", ""),
source = source,
)
},
description = doc.select("span.story-desc").html(),
state = when (doc.select("div.section-status:contains(Status) span").last()?.text()) {
"Ongoing" -> MangaState.ONGOING
"Completed" -> MangaState.FINISHED
else -> null
},
chapters = getChapters(manga.url, doc),
)
}
private fun getChapters(mangaUrl: String, doc: Document): List<MangaChapter> {
val dateFormat = SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSSSSS'Z'", sourceLocale)
val script = doc.selectFirstOrThrow("script:containsData(chapters:)")
val json =
script.data().substringAfter("chapters:[").substringBeforeLast(')').substringBefore("],latest_chapter:")
.split("},")
val slug = mangaUrl.substringAfterLast("/")
val chapter = ArrayList<MangaChapter>()
var lastIndexed = 0
json.mapIndexed { i, t ->
if (t.contains("Chapter")) {
val id = t.substringAfter("id:").substringBefore(",created_at")
val url = "/reading/$slug/$id"
val date = t.substringAfter("created_at:\"").substringBefore("\"")
val name = t.substringAfter("name:\"").substringBefore("\"")
chapter.add(
MangaChapter(
id = generateUid(url),
name = name,
number = i + 1,
url = url,
uploadDate = dateFormat.tryParse(date),
source = source,
scanlator = null,
branch = null,
),
)
lastIndexed = i
}
}
// last chapter
val id = script.data().substringAfter("Sign in\",").substringBefore(",\"").split(",").last()
val url = "/reading/$slug/$id"
val date = script.data().substringAfter("$id,\"").substringBefore("\",")
val name = script.data().substringAfter("$date\",\"").substringBefore("\",")
chapter.add(
MangaChapter(
id = generateUid(url),
name = name,
number = lastIndexed + 1,
url = url,
uploadDate = dateFormat.tryParse(date),
source = source,
scanlator = null,
branch = null,
),
)
return chapter
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val id = chapter.url.substringAfterLast("/")
val json = webClient.httpGet("https://api.mangaowl.to/v1/chapters/$id/images?page_size=100").parseJson()
return json.getJSONArray("results").mapJSON { jo ->
MangaPage(
id = generateUid(jo.getString("image")),
preview = null,
source = chapter.source,
url = jo.getString("image"),
)
}
}
}

View File

@@ -0,0 +1,194 @@
package org.koitharu.kotatsu.parsers.site.en
import androidx.collection.ArrayMap
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.util.*
@MangaSourceParser("MANHWA18", "Manhwa18", "en", type = ContentType.HENTAI)
class Manhwa18Parser(context: MangaLoaderContext) :
PagedMangaParser(context, MangaSource.MANHWA18, pageSize = 18, searchPageSize = 18) {
override val configKeyDomain: ConfigKey.Domain = ConfigKey.Domain("manhwa18.net")
override val availableSortOrders: Set<SortOrder>
get() = EnumSet.of(SortOrder.UPDATED, SortOrder.POPULARITY, SortOrder.ALPHABETICAL, SortOrder.NEWEST)
private val tagsMap = SuspendLazy(::parseTags)
override suspend fun getFavicons(): Favicons {
return Favicons(
listOf(
Favicon("https://$domain/uploads/logos/logo-mini.png", 92, null),
),
domain,
)
}
override suspend fun getDetails(manga: Manga): Manga {
val docs = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
val cardInfoElement = docs.selectFirst("div.series-information")
val author = cardInfoElement?.selectFirst(".info-name:contains(Author)")?.parent()
?.select("a")
?.joinToString(", ") { it.text() }
val availableTags = tagsMap.get()
val tags = cardInfoElement?.selectFirst(".info-name:contains(Genre)")?.parent()
?.select("a")
?.mapNotNullToSet { availableTags[it.text().lowercase(Locale.ENGLISH)] }
val state = cardInfoElement?.selectFirst(".info-name:contains(Status)")?.parent()
?.selectFirst("a")
?.let {
when (it.text().lowercase()) {
"on going" -> MangaState.ONGOING
"completed" -> MangaState.FINISHED
else -> null
}
}
return manga.copy(
altTitle = cardInfoElement?.selectFirst("b:contains(Other names)")?.parent()?.ownText()?.removePrefix(": "),
author = author,
description = docs.selectFirst(".series-summary .summary-content")?.html(),
tags = tags.orEmpty(),
state = state,
chapters = docs.select(".card-body > .list-chapters > a").mapChapters(reversed = true) { index, element ->
val chapterUrl = element.attrAsAbsoluteUrlOrNull("href")?.toRelativeUrl(domain)
?: return@mapChapters null
val uploadDate = parseUploadDate(element.selectFirst(".chapter-time")?.text())
MangaChapter(
id = generateUid(chapterUrl),
name = element.selectFirst(".chapter-name")?.text().orEmpty(),
number = index + 1,
url = chapterUrl,
scanlator = null,
uploadDate = uploadDate,
branch = null,
source = MangaSource.MANHWA18,
)
},
)
}
// 7 minutes ago
// 5 hours ago
// 2 days ago
// 2 weeks ago
// 4 years ago
private fun parseUploadDate(timeStr: String?): Long {
timeStr ?: return 0
val timeWords = timeStr.split(' ')
if (timeWords.size != 3) return 0
val timeWord = timeWords[1]
val timeAmount = timeWords[0].toIntOrNull() ?: return 0
val timeUnit = when (timeWord) {
"minute", "minutes" -> Calendar.MINUTE
"hour", "hours" -> Calendar.HOUR
"day", "days" -> Calendar.DAY_OF_YEAR
"week", "weeks" -> Calendar.WEEK_OF_YEAR
"month", "months" -> Calendar.MONTH
"year", "years" -> Calendar.YEAR
else -> return 0
}
val cal = Calendar.getInstance()
cal.add(timeUnit, -timeAmount)
return cal.time.time
}
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val sortQuery = when (sortOrder) {
SortOrder.ALPHABETICAL -> "az"
SortOrder.POPULARITY -> "top"
SortOrder.UPDATED -> "update"
SortOrder.NEWEST -> "new"
else -> ""
}
val tagQuery = tags?.joinToString(",") { it.key }.orEmpty()
val url = buildString {
append("https://")
append(domain)
append("/tim-kiem?page=")
append(page)
if (!query.isNullOrEmpty()) {
append("&q=")
append(query.urlEncoded())
}
append("&accept_genres=$tagQuery")
append("&sort=")
append(sortQuery)
}
val docs = webClient.httpGet(url).parseHtml()
return docs.select(".card-body .thumb-item-flow")
.map {
val titleElement = it.selectFirstOrThrow(".thumb_attr.series-title > a")
val absUrl = titleElement.attrAsAbsoluteUrl("href")
Manga(
id = generateUid(absUrl.toRelativeUrl(domain)),
title = titleElement.text(),
altTitle = null,
url = absUrl.toRelativeUrl(domain),
publicUrl = absUrl,
rating = RATING_UNKNOWN,
isNsfw = true,
coverUrl = it.selectFirst("div.img-in-ratio")?.attrAsAbsoluteUrl("data-bg").orEmpty(),
tags = emptySet(),
state = null,
author = null,
largeCoverUrl = null,
description = null,
source = MangaSource.MANHWA18,
)
}
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val chapterUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(chapterUrl).parseHtml()
return doc.requireElementById("chapter-content").select("img").mapNotNull {
val url = it.attrAsRelativeUrlOrNull("data-src")
?: it.attrAsRelativeUrlOrNull("src")
?: return@mapNotNull null
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = MangaSource.MANHWA18,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
return tagsMap.get().values.toSet()
}
private suspend fun parseTags(): Map<String, MangaTag> {
val doc = webClient.httpGet("https://$domain/tim-kiem?q=").parseHtml()
val list = doc.getElementsByAttribute("data-genre-id")
if (list.isEmpty()) {
return emptyMap()
}
val result = ArrayMap<String, MangaTag>(list.size)
for (item in list) {
val id = item.attr("data-genre-id")
val name = item.text()
result[name.lowercase(Locale.ENGLISH)] = MangaTag(
title = name.toTitleCase(Locale.ENGLISH),
key = id,
source = source,
)
}
return result
}
}

View File

@@ -0,0 +1,151 @@
package org.koitharu.kotatsu.parsers.site.en
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.text.DateFormat
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("MANHWASMEN", "ManhwasMen", "en", type = ContentType.HENTAI)
class ManhwasMen(context: MangaLoaderContext) :
PagedMangaParser(context, MangaSource.MANHWASMEN, pageSize = 30, searchPageSize = 30) {
override val configKeyDomain: ConfigKey.Domain = ConfigKey.Domain("manhwas.men")
override val isMultipleTagsSupported = false
override val availableSortOrders: Set<SortOrder>
get() = EnumSet.of(SortOrder.POPULARITY)
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val tag = tags.oneOrThrowIfMany()
val url = buildString {
append("https://")
append(domain)
append("/manga-list")
append("?page=")
append(page)
when {
!query.isNullOrEmpty() -> {
append("&search=")
append(query.urlEncoded())
}
!tags.isNullOrEmpty() -> {
append("&genero=")
append(tag?.key.orEmpty())
}
}
}
val doc = webClient.httpGet(url).parseHtml()
return doc.select("ul.animes li").map { li ->
val href = li.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(domain),
coverUrl = li.selectFirst("img")?.src().orEmpty(),
title = li.selectFirst(".title")?.text().orEmpty(),
altTitle = null,
rating = RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = null,
source = source,
isNsfw = isNsfwSource,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
val tags = webClient.httpGet("https://$domain/manga-list").parseHtml()
.selectLastOrThrow(".filter-bx .form-group select.custom-select").select("option").drop(1)
return tags.mapNotNullToSet { option ->
MangaTag(
key = option.attr("value").substringAfterLast("="),
title = option.text(),
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga {
val doc = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
return manga.copy(
tags = doc.body().select(".genres a").mapNotNullToSet { a ->
MangaTag(
key = a.attr("href").substringAfterLast('='),
title = a.text(),
source = source,
)
},
description = doc.select(".sinopsis").html(),
state = when (doc.selectLast(".anime-type-peli")?.text()?.lowercase()) {
"ongoing" -> MangaState.ONGOING
"completed" -> MangaState.FINISHED
else -> null
},
chapters = doc.select(".episodes-list li").mapChapters(reversed = true) { i, li ->
val url = li.selectFirstOrThrow("a").attrAsRelativeUrl("href")
MangaChapter(
id = generateUid(url),
name = li.selectFirstOrThrow(".flex-grow-1 span").text(),
number = i + 1,
url = url,
scanlator = null,
uploadDate = parseChapterDate(
SimpleDateFormat("dd/MM/yyyy", sourceLocale),
li.selectLastOrThrow(".flex-grow-1 span").text(),
),
branch = null,
source = source,
)
},
)
}
private fun parseChapterDate(dateFormat: DateFormat, date: String?): Long {
val d = date?.lowercase() ?: return 0
return when {
d.endsWith(" ago") -> parseRelativeDate(date)
else -> dateFormat.tryParse(date)
}
}
private fun parseRelativeDate(date: String): Long {
val number = Regex("""(\d+)""").find(date)?.value?.toIntOrNull() ?: return 0
val cal = Calendar.getInstance()
return when {
WordSet("second").anyWordIn(date) -> cal.apply { add(Calendar.SECOND, -number) }.timeInMillis
WordSet("minute", "minutes").anyWordIn(date) -> cal.apply { add(Calendar.MINUTE, -number) }.timeInMillis
WordSet("hour", "hours").anyWordIn(date) -> cal.apply { add(Calendar.HOUR, -number) }.timeInMillis
WordSet("day", "days").anyWordIn(date) -> cal.apply { add(Calendar.DAY_OF_MONTH, -number) }.timeInMillis
WordSet("week", "weeks").anyWordIn(date) -> cal.apply { add(Calendar.WEEK_OF_YEAR, -number) }.timeInMillis
WordSet("month", "months").anyWordIn(date) -> cal.apply { add(Calendar.MONTH, -number) }.timeInMillis
WordSet("year").anyWordIn(date) -> cal.apply { add(Calendar.YEAR, -number) }.timeInMillis
else -> 0
}
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val doc = webClient.httpGet(chapter.url.toAbsoluteUrl(domain)).parseHtml().requireElementById("chapter_imgs")
return doc.select("img").map { img ->
val url = img.src()?.toRelativeUrl(domain) ?: img.parseFailed("Image src not found")
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
)
}
}
}

View File

@@ -0,0 +1,96 @@
package org.koitharu.kotatsu.parsers.site.en
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaParser
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("PO2SCANS", "Po2Scans", "en")
internal class Po2Scans(context: MangaLoaderContext) : MangaParser(context, MangaSource.PO2SCANS) {
override val availableSortOrders: Set<SortOrder> = EnumSet.of(SortOrder.ALPHABETICAL)
override val configKeyDomain = ConfigKey.Domain("po2scans.com")
override suspend fun getList(offset: Int, query: String?, tags: Set<MangaTag>?, sortOrder: SortOrder): List<Manga> {
if (offset > 0) {
return emptyList()
}
val url = buildString {
append("https://$domain/series")
if (!query.isNullOrEmpty()) {
append("?search=")
append(query.urlEncoded())
}
}
val doc = webClient.httpGet(url).parseHtml()
return doc.select(".series-list").map { div ->
val href = div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
title = div.selectFirstOrThrow("h2").text(),
altTitle = null,
url = href,
publicUrl = href.toAbsoluteUrl(domain),
rating = RATING_UNKNOWN,
isNsfw = false,
coverUrl = div.selectFirstOrThrow("img").src()?.toAbsoluteUrl(domain).orEmpty(),
tags = emptySet(),
state = null,
author = null,
source = source,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> = emptySet()
override suspend fun getDetails(manga: Manga): Manga {
val doc = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
val dateFormat = SimpleDateFormat("dd MMM, yy", Locale.ENGLISH)
return manga.copy(
altTitle = null,
state = when (doc.select(".status span").last()?.text()) {
"Ongoing" -> MangaState.ONGOING
"Done" -> MangaState.FINISHED
else -> null
},
tags = emptySet(),
author = doc.select(".author span").last()?.text(),
description = doc.selectFirstOrThrow(".summary").text(),
chapters = doc.select(".chap-section .chap")
.mapChapters(reversed = true) { i, div ->
val a = div.selectFirstOrThrow("a")
val url = "/" + a.attrAsRelativeUrl("href").toAbsoluteUrl(domain)
MangaChapter(
id = generateUid(url),
name = a.text(),
number = i + 1,
url = url,
scanlator = null,
uploadDate = dateFormat.tryParse(div.select(".detail span").last()?.text()),
branch = null,
source = source,
)
},
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
return doc.select(".swiper-slide img").map { img ->
val url = img.src()?.replace("./assets", "/assets")?.toRelativeUrl(domain)
?: img.parseFailed("Image src not found")
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
)
}
}
}

View File

@@ -0,0 +1,170 @@
package org.koitharu.kotatsu.parsers.site.en
import androidx.collection.ArraySet
import kotlinx.coroutines.async
import kotlinx.coroutines.awaitAll
import kotlinx.coroutines.coroutineScope
import org.jsoup.nodes.Element
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.util.*
@MangaSourceParser("PURURIN", "Pururin", "en", ContentType.HENTAI)
internal class Pururin(context: MangaLoaderContext) :
PagedMangaParser(context, MangaSource.PURURIN, pageSize = 20) {
override val availableSortOrders: Set<SortOrder> =
EnumSet.of(SortOrder.UPDATED, SortOrder.POPULARITY, SortOrder.RATING, SortOrder.ALPHABETICAL)
override val configKeyDomain = ConfigKey.Domain("pururin.to")
override val isMultipleTagsSupported = false
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val tag = tags.oneOrThrowIfMany()
val url = buildString {
append("https://")
append(domain)
if (!query.isNullOrEmpty()) {
append("/search?q=")
append(query.urlEncoded())
append("&page=")
append(page)
} else {
append("/browse")
if (!tags.isNullOrEmpty()) {
append("/tags/content/")
append(tag?.key.orEmpty())
append("/")
}
append("?page=")
append(page)
append("&sort=")
when (sortOrder) {
SortOrder.UPDATED -> append("")
SortOrder.POPULARITY -> append("most-viewed")
SortOrder.RATING -> append("highest-rated")
SortOrder.ALPHABETICAL -> append("title")
else -> append("")
}
}
}
val doc = webClient.httpGet(url).parseHtml()
return doc.select(".row-gallery a.card-gallery").map { a ->
val href = a.attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(domain),
coverUrl = a.selectFirst("img.card-img-top")?.src().orEmpty(),
title = a.selectFirst(".title")?.text().orEmpty(),
altTitle = null,
rating = RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = null,
source = source,
isNsfw = isNsfwSource,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
return coroutineScope {
(1..4).map { page ->
async { getTags(page) }
}
}.awaitAll().flattenTo(ArraySet(360))
}
private suspend fun getTags(page: Int): Set<MangaTag> {
val url = "https://$domain/tags/content?order=uses&page=$page"
val root = webClient.httpGet(url).parseHtml()
return root.parseTags()
}
private fun Element.parseTags() = select("table tr td a").mapToSet {
val href = it.attr("href").substringAfterLast("content/").substringBeforeLast('/')
MangaTag(
key = href,
title = it.text(),
source = source,
)
}
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val fullUrl = manga.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
manga.copy(
description = doc.selectFirst("p.mb-2")?.text().orEmpty(),
rating = doc.selectFirst("td span.rating")?.attr("content")?.toFloatOrNull()?.div(5f) ?: RATING_UNKNOWN,
tags = doc.body().select("tr:contains(Contents) ul.list-inline a").mapNotNullToSet {
val href = it.attr("href").substringAfterLast("content/").substringBeforeLast('/')
MangaTag(
key = href,
title = it.text(),
source = source,
)
},
author = doc.selectFirst("a[itemprop=author]")?.text(),
chapters = listOf(
MangaChapter(
id = manga.id,
name = manga.title,
number = 1,
url = manga.url,
scanlator = null,
uploadDate = 0,
branch = null,
source = source,
),
),
)
}
override suspend fun getRelatedManga(seed: Manga): List<Manga> {
val doc = webClient.httpGet(seed.url.toAbsoluteUrl(domain)).parseHtml()
val root = doc.body().selectFirstOrThrow(".row-gallery-small")
return root.select("a.card-gallery").mapNotNull { a ->
val href = a.attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(domain),
coverUrl = a.selectFirst("img.card-img-top")?.src().orEmpty(),
title = a.selectFirst(".title")?.text().orEmpty(),
altTitle = null,
rating = RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = null,
source = source,
isNsfw = false,
)
}
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
return doc.select(".gallery-preview img").map { url ->
val img = url.src()?.toRelativeUrl(domain) ?: url.parseFailed("Image src not found")
val urlImage = img.replace("t.", ".")
MangaPage(
id = generateUid(urlImage),
url = urlImage,
preview = null,
source = source,
)
}
}
}

View File

@@ -0,0 +1,132 @@
package org.koitharu.kotatsu.parsers.site.es
import kotlinx.coroutines.coroutineScope
import okhttp3.Headers
import org.jsoup.nodes.Document
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.network.UserAgents
import org.koitharu.kotatsu.parsers.util.*
import java.util.*
@MangaSourceParser("TEMPLESCANESP", "TempleScanEsp", "es", ContentType.HENTAI)
internal class TempleScanEsp(context: MangaLoaderContext) :
PagedMangaParser(context, MangaSource.TEMPLESCANESP, pageSize = 15) {
override val availableSortOrders: Set<SortOrder> = EnumSet.of(SortOrder.NEWEST, SortOrder.UPDATED)
override val configKeyDomain = ConfigKey.Domain("templescanesp.net")
override val headers: Headers = Headers.Builder()
.add("User-Agent", UserAgents.CHROME_DESKTOP)
.build()
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val url = buildString {
append("https://")
append(domain)
if (sortOrder == SortOrder.NEWEST) {
append("/comics")
append("?page=")
append(page.toString())
} else {
if (page > 1) {
return emptyList()
}
}
}
val doc = webClient.httpGet(url).parseHtml()
return doc.select("div.grid figure").ifEmpty {
doc.requireElementById("projectsDiv").select("figure")
}.map { div ->
val href = div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(div.host ?: domain),
coverUrl = div.selectFirst("img")?.src().orEmpty(),
title = div.selectFirst("figcaption")?.text().orEmpty(),
altTitle = null,
rating = RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = null,
source = source,
isNsfw = isNsfwSource,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> = emptySet()
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val fullUrl = manga.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
val chaptersDeferred = getChapters(doc)
manga.copy(
description = doc.requireElementById("section-sinopsis").html(),
chapters = chaptersDeferred,
)
}
private fun getChapters(doc: Document): List<MangaChapter> {
return doc.body().select("div.grid-capitulos div.contenedor-capitulo-miniatura")
.mapChapters(reversed = true) { i, div ->
val href = div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
val date = parseUploadDate(div.selectFirstOrThrow("time").text())
MangaChapter(
id = generateUid(href),
name = div.requireElementById("name").text(),
number = i + 1,
url = href,
uploadDate = date,
source = source,
scanlator = null,
branch = null,
)
}
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
return doc.select("main.contenedor-imagen img").map { url ->
val img = url.src()?.toRelativeUrl(domain) ?: url.parseFailed("Image src not found")
MangaPage(
id = generateUid(img),
url = img,
preview = null,
source = source,
)
}
}
private fun parseUploadDate(timeStr: String?): Long {
timeStr ?: return 0
val timeWords = timeStr.split(' ')
if (timeWords.size != 3) return 0
val timeWord = timeWords[1]
val timeAmount = timeWords[0].toIntOrNull() ?: return 0
val timeUnit = when (timeWord) {
"minute", "minutes" -> Calendar.MINUTE
"hour", "hours" -> Calendar.HOUR
"day", "days" -> Calendar.DAY_OF_YEAR
"week", "weeks" -> Calendar.WEEK_OF_YEAR
"month", "months" -> Calendar.MONTH
"year", "years" -> Calendar.YEAR
else -> return 0
}
val cal = Calendar.getInstance()
cal.add(timeUnit, -timeAmount)
return cal.time.time
}
}

View File

@@ -0,0 +1,227 @@
package org.koitharu.kotatsu.parsers.site.es
import okhttp3.Headers
import okhttp3.HttpUrl.Companion.toHttpUrl
import org.jsoup.nodes.Document
import org.jsoup.nodes.Element
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("TUMANGAONLINE", "TuMangaOnline", "es")
class TuMangaOnlineParser(context: MangaLoaderContext) : PagedMangaParser(
context,
source = MangaSource.TUMANGAONLINE,
pageSize = 24,
) {
override val configKeyDomain = ConfigKey.Domain("visortmo.com")
private val chapterDateFormat = SimpleDateFormat("yyyy-MM-dd", sourceLocale)
override val availableSortOrders: Set<SortOrder> = EnumSet.of(
SortOrder.ALPHABETICAL,
SortOrder.UPDATED,
SortOrder.NEWEST,
SortOrder.POPULARITY,
SortOrder.RATING,
)
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val order =
when (sortOrder) {
SortOrder.POPULARITY -> "likes_count"
SortOrder.UPDATED -> "release_date"
SortOrder.NEWEST -> "creation"
SortOrder.ALPHABETICAL -> "alphabetically"
SortOrder.RATING -> "score"
}
val url = buildString {
append("/library")
if (query.isNullOrEmpty()) {
append("?order_item=")
append(order)
append("&order_dir=desc")
append("&filter_by=title")
if (tags != null) {
for (tag in tags) {
append("&genders[]=${tag.key}")
}
}
} else {
append("?title=$query")
}
append("&_pg=1")
append("&page=$page")
}.toAbsoluteUrl(domain)
val doc = webClient.httpGet(url, headers).parseHtml()
val items = doc.body().select("div.element")
return items.mapNotNull { item ->
val href =
item.selectFirst("a")?.attrAsRelativeUrlOrNull("href")?.substringAfter(' ') ?: return@mapNotNull null
Manga(
id = generateUid(href),
title = item.selectFirst("h4.text-truncate")?.text() ?: return@mapNotNull null,
coverUrl = item.select("style").toString().substringAfter("('").substringBeforeLast("')"),
altTitle = null,
author = null,
rating = item.selectFirst("span.score")?.text()?.toFloatOrNull()?.div(10F) ?: RATING_UNKNOWN,
url = href,
isNsfw = item.select("i").hasClass("fas fa-heartbeat fa-2x"),
tags = emptySet(),
state = null,
publicUrl = href.toAbsoluteUrl(doc.host ?: domain),
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga {
val doc = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
val contents = doc.body().selectFirstOrThrow("section.element-header-content")
return manga.copy(
description = contents.selectFirst("p.element-description")?.html(),
largeCoverUrl = contents.selectFirst(".book-thumbnail")?.attrAsAbsoluteUrlOrNull("src"),
state = parseStatus(contents.select("span.book-status").text().orEmpty()),
author = contents.selectFirst("h5.card-title")?.attr("title")?.substringAfter(", "),
chapters = if (doc.select("div.chapters").isEmpty()) {
doc.select(oneShotChapterListSelector).mapChapters(reversed = true) { _, item ->
oneShotChapterFromElement(item)
}
} else {
val chapters = ChaptersListBuilder(10)
doc.select(regularChapterListSelector).reversed().forEachIndexed { i, item ->
val chaptername = item.select("div.col-10.text-truncate").text().replace("&nbsp;", " ").trim()
val scanelement = item.select("ul.chapter-list > li")
scanelement.forEach { chapters.add(regularChapterFromElement(it, chaptername, i)) }
}
chapters.toList()
},
)
}
private val oneShotChapterListSelector = "div.chapter-list-element > ul.list-group li.list-group-item"
private fun oneShotChapterFromElement(element: Element): MangaChapter {
val href = element.selectFirstOrThrow("div.row > .text-right > a").attrAsRelativeUrl("href")
return MangaChapter(
id = generateUid(href),
name = "One Shot",
number = 1,
url = href,
scanlator = element.select("div.col-md-6.text-truncate").text(),
branch = null,
uploadDate = chapterDateFormat.tryParse(element.select("span.badge.badge-primary.p-2").first()?.text()),
source = source,
)
}
private val regularChapterListSelector = "div.chapters > ul.list-group li.p-0.list-group-item"
private fun regularChapterFromElement(element: Element, chName: String, number: Int): MangaChapter {
val href = element.selectFirstOrThrow("div.row > .text-right > a").attrAsRelativeUrl("href")
return MangaChapter(
id = generateUid(href),
name = chName,
number = number + 1,
url = href,
scanlator = element.select("div.col-md-6.text-truncate").text(),
branch = null,
uploadDate = chapterDateFormat.tryParse(element.select("span.badge.badge-primary.p-2").first()?.text()),
source = source,
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val redirectDoc = webClient.httpGet(chapter.url.toAbsoluteUrl(domain), headers).parseHtml()
var doc = redirectToReadingPage(redirectDoc)
val currentUrl = doc.location()
val newUrl = if (!currentUrl.contains("cascade")) {
currentUrl.substringBefore("paginated") + "cascade"
} else {
currentUrl
}
if (currentUrl != newUrl) {
doc = webClient.httpGet(newUrl, headers).parseHtml()
}
return doc.select("div.viewer-container img:not(noscript img)").map {
val href = if (it.hasAttr("data-src")) {
it.attr("abs:data-src")
} else {
it.attr("abs:src")
}
MangaPage(
id = generateUid(href),
url = href,
preview = null,
source = source,
)
}
}
private suspend fun redirectToReadingPage(document: Document): Document {
val script1 = document.selectFirst("script:containsData(uniqid)")
val script2 = document.selectFirst("script:containsData(window.location.replace)")
val redirectHeaders = Headers.Builder().set("Referer", document.baseUri()).build()
if (script1 != null) {
val data = script1.data()
val regexParams = """\{uniqid:'(.+)',cascade:(.+)\}""".toRegex()
val regexAction = """form\.action\s?=\s?'(.+)'""".toRegex()
val params = regexParams.find(data)!!
val action = regexAction.find(data)!!.groupValues[1].toHttpUrl()
val formBody = mapOf(
"uniqid" to params.groupValues[1],
"cascade" to params.groupValues[2],
)
return redirectToReadingPage(webClient.httpPost(action, formBody, redirectHeaders).parseHtml())
}
if (script2 != null) {
val data = script2.data()
val regexRedirect = """window\.location\.replace\('(.+)'\)""".toRegex()
val url = regexRedirect.find(data)!!.groupValues[1]
return redirectToReadingPage(webClient.httpGet(url, redirectHeaders).parseHtml())
}
return document
}
override suspend fun getAvailableTags(): Set<MangaTag> {
val doc = webClient.httpGet("https://$domain/library", headers).parseHtml()
val elements = doc.body().select("div#books-genders > div > div")
return elements.mapNotNullToSet { element ->
MangaTag(
title = element.select("label").text(),
key = element.select("input").attr("value"),
source = source,
)
}
}
private fun parseStatus(status: String) = when {
status.contains("Publicándose") -> MangaState.ONGOING
status.contains("Finalizado") -> MangaState.FINISHED
else -> null
}
}

View File

@@ -0,0 +1,294 @@
package org.koitharu.kotatsu.parsers.site.fmreader
import kotlinx.coroutines.async
import kotlinx.coroutines.coroutineScope
import org.jsoup.nodes.Document
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.text.DateFormat
import java.text.SimpleDateFormat
import java.util.*
internal abstract class FmreaderParser(
context: MangaLoaderContext,
source: MangaSource,
domain: String,
pageSize: Int = 20,
) : PagedMangaParser(context, source, pageSize) {
override val configKeyDomain = ConfigKey.Domain(domain)
override val availableSortOrders: Set<SortOrder> = EnumSet.of(
SortOrder.UPDATED,
SortOrder.POPULARITY,
SortOrder.ALPHABETICAL,
)
override val isMultipleTagsSupported = false
protected open val listUrl = "/manga-list.html"
protected open val datePattern = "MMMM d, yyyy"
protected open val tagPrefix = "manga-list-genre-"
init {
paginator.firstPage = 1
searchPaginator.firstPage = 1
}
@JvmField
protected val ongoing: Set<String> = setOf(
"On going",
"Incomplete",
"En curso",
)
@JvmField
protected val finished: Set<String> = setOf(
"Completed",
"Completado",
)
@JvmField
protected val abandoned: Set<String> = hashSetOf(
"Canceled",
"Cancelled",
"Drop",
)
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val tag = tags.oneOrThrowIfMany()
val url = buildString {
append("https://")
append(domain)
append(listUrl)
append("?page=")
append(page.toString())
when {
!query.isNullOrEmpty() -> {
append("&name=")
append(query.urlEncoded())
}
!tags.isNullOrEmpty() -> {
append("&genre=")
append(tag?.key.orEmpty())
}
}
append("&sort=")
when (sortOrder) {
SortOrder.POPULARITY -> append("views")
SortOrder.UPDATED -> append("last_update")
SortOrder.ALPHABETICAL -> append("name")
else -> append("last_update")
}
}
val doc = webClient.httpGet(url).parseHtml()
return doc.select("div.thumb-item-flow").map { div ->
val href = div.selectFirstOrThrow("div.series-title a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(div.host ?: domain),
coverUrl = div.selectFirstOrThrow("div.img-in-ratio").attr("data-bg")
?: div.selectFirstOrThrow("div.img-in-ratio").attr("style").substringAfter("('")
.substringBeforeLast("')"),
title = div.selectFirstOrThrow("div.series-title").text().orEmpty(),
altTitle = null,
rating = RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = null,
source = source,
isNsfw = isNsfwSource,
)
}
}
protected open val selectBodyTag = "ul.filter-type li a"
override suspend fun getAvailableTags(): Set<MangaTag> {
val doc = webClient.httpGet("https://$domain/$listUrl").parseHtml()
return doc.select(selectBodyTag).mapNotNullToSet { a ->
val href = a.attr("href").substringAfter(tagPrefix).substringBeforeLast(".html")
MangaTag(
key = href,
title = a.text(),
source = source,
)
}
}
protected open val selectDesc = "div.summary-content"
protected open val selectState = "ul.manga-info li:contains(Status) a"
protected open val selectAlt = "ul.manga-info li:contains(Other names)"
protected open val selectAut = "ul.manga-info li:contains(Author(s)) a"
protected open val selectTag = "ul.manga-info li:contains(Genre(s)) a"
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val fullUrl = manga.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
val chaptersDeferred = async { getChapters(doc) }
val desc = doc.selectFirst(selectDesc)?.html()
val stateDiv = doc.selectFirst(selectState)
val state = stateDiv?.let {
when (it.text()) {
in ongoing -> MangaState.ONGOING
in finished -> MangaState.FINISHED
in abandoned -> MangaState.ABANDONED
else -> null
}
}
val alt = doc.body().selectFirst(selectAlt)?.text()?.replace("Other names", "")
val auth = doc.body().selectFirst(selectAut)?.text()
manga.copy(
tags = doc.body().select(selectTag).mapNotNullToSet { a ->
MangaTag(
key = a.attr("href").substringAfter(tagPrefix).substringBeforeLast(".html"),
title = a.text().toTitleCase(),
source = source,
)
},
description = desc,
altTitle = alt,
author = auth,
state = state,
chapters = chaptersDeferred.await(),
)
}
protected open val selectDate = "div.chapter-time"
protected open val selectChapter = "ul.list-chapters a"
protected open suspend fun getChapters(doc: Document): List<MangaChapter> {
val dateFormat = SimpleDateFormat(datePattern, sourceLocale)
return doc.body().select(selectChapter).mapChapters(reversed = true) { i, a ->
val href = a.attrAsRelativeUrl("href")
val dateText = a.selectFirst(selectDate)?.text()
MangaChapter(
id = generateUid(href),
name = a.selectFirstOrThrow("div.chapter-name").text(),
number = i + 1,
url = href,
uploadDate = parseChapterDate(
dateFormat,
dateText,
),
source = source,
scanlator = null,
branch = null,
)
}
}
protected open val selectPage = "div.chapter-content img"
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
return doc.select(selectPage).map { img ->
val url = img.src()?.toRelativeUrl(domain) ?: img.parseFailed("Image src not found")
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
)
}
}
protected fun parseChapterDate(dateFormat: DateFormat, date: String?): Long {
// Clean date (e.g. 5th December 2019 to 5 December 2019) before parsing it
val d = date?.lowercase() ?: return 0
return when {
d.endsWith(" ago") ||
d.endsWith(" atrás") ||
// short Hours
d.endsWith(" h") ||
// short Day
d.endsWith(" d") -> parseRelativeDate(date)
// Handle 'yesterday' and 'today', using midnight
d.startsWith("year") -> Calendar.getInstance().apply {
add(Calendar.DAY_OF_MONTH, -1) // yesterday
set(Calendar.HOUR_OF_DAY, 0)
set(Calendar.MINUTE, 0)
set(Calendar.SECOND, 0)
set(Calendar.MILLISECOND, 0)
}.timeInMillis
d.startsWith("today") -> Calendar.getInstance().apply {
set(Calendar.HOUR_OF_DAY, 0)
set(Calendar.MINUTE, 0)
set(Calendar.SECOND, 0)
set(Calendar.MILLISECOND, 0)
}.timeInMillis
date.contains(Regex("""\d(st|nd|rd|th)""")) -> date.split(" ").map {
if (it.contains(Regex("""\d\D\D"""))) {
it.replace(Regex("""\D"""), "")
} else {
it
}
}.let { dateFormat.tryParse(it.joinToString(" ")) }
else -> dateFormat.tryParse(date)
}
}
// Parses dates in this form:
// 21 hours ago
private fun parseRelativeDate(date: String): Long {
val number = Regex("""(\d+)""").find(date)?.value?.toIntOrNull() ?: return 0
val cal = Calendar.getInstance()
return when {
WordSet("second").anyWordIn(date) -> cal.apply { add(Calendar.SECOND, -number) }.timeInMillis
WordSet("min", "minute", "minutes", "minuto", "minutos").anyWordIn(date) -> cal.apply {
add(
Calendar.MINUTE,
-number,
)
}.timeInMillis
WordSet("hour", "hours", "hora", "horas", "h").anyWordIn(date) -> cal.apply {
add(
Calendar.HOUR,
-number,
)
}.timeInMillis
WordSet("day", "days", "día", "dia").anyWordIn(date) -> cal.apply {
add(
Calendar.DAY_OF_MONTH,
-number,
)
}.timeInMillis
WordSet("week", "weeks", "semana", "semanas").anyWordIn(date) -> cal.apply {
add(
Calendar.WEEK_OF_YEAR,
-number,
)
}.timeInMillis
WordSet("month", "months", "mes", "meses").anyWordIn(date) -> cal.apply {
add(
Calendar.MONTH,
-number,
)
}.timeInMillis
WordSet("year", "año", "años").anyWordIn(date) -> cal.apply { add(Calendar.YEAR, -number) }.timeInMillis
else -> 0
}
}
}

View File

@@ -0,0 +1,152 @@
package org.koitharu.kotatsu.parsers.site.fmreader.en
import kotlinx.coroutines.async
import kotlinx.coroutines.coroutineScope
import org.jsoup.nodes.Document
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.site.fmreader.FmreaderParser
import org.koitharu.kotatsu.parsers.util.*
import java.text.SimpleDateFormat
@MangaSourceParser("MANHWA18COM", "Manhwa18.com", "en", ContentType.HENTAI)
internal class Manhwa18Com(context: MangaLoaderContext) :
FmreaderParser(context, MangaSource.MANHWA18COM, "manhwa18.com") {
override val listUrl = "/tim-kiem"
override val selectState = "div.info-item:contains(Status) span.info-value "
override val selectAlt = "div.info-item:contains(Other name) span.info-value "
override val selectTag = "div.info-item:contains(Genre) span.info-value a"
override val datePattern = "dd/MM/yyyy"
override val selectPage = "div#chapter-content img"
override val selectBodyTag = "div.genres-menu a"
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val tag = tags.oneOrThrowIfMany()
val url = buildString {
append("https://")
append(domain)
if (!tags.isNullOrEmpty()) {
append("/genre/")
append(tag?.key.orEmpty())
append("?page=")
append(page.toString())
append("&sort=")
when (sortOrder) {
SortOrder.POPULARITY -> append("views")
SortOrder.UPDATED -> append("last_update")
SortOrder.ALPHABETICAL -> append("name")
else -> append("last_update")
}
} else {
append(listUrl)
append("?page=")
append(page.toString())
when {
!query.isNullOrEmpty() -> {
append("&q=")
append(query.urlEncoded())
}
}
append("&sort=")
when (sortOrder) {
SortOrder.POPULARITY -> append("views")
SortOrder.UPDATED -> append("last_update")
SortOrder.ALPHABETICAL -> append("name")
else -> append("last_update")
}
}
}
val doc = webClient.httpGet(url).parseHtml()
return doc.select("div.thumb-item-flow").map { div ->
val href = div.selectFirstOrThrow("div.series-title a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(div.host ?: domain),
coverUrl = div.selectFirstOrThrow("div.img-in-ratio").attr("data-bg")
?: div.selectFirstOrThrow("div.img-in-ratio").attr("style").substringAfter("('")
.substringBeforeLast("')"),
title = div.selectFirstOrThrow("div.series-title").text().orEmpty(),
altTitle = null,
rating = RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = null,
source = source,
isNsfw = isNsfwSource,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
val doc = webClient.httpGet("https://$domain/$listUrl").parseHtml()
return doc.select(selectBodyTag).mapNotNullToSet { a ->
val href = a.attr("href").substringAfterLast("/")
MangaTag(
key = href,
title = a.text(),
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val fullUrl = manga.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
val chaptersDeferred = async { getChapters(doc) }
val desc = doc.selectFirstOrThrow(selectDesc).html()
val stateDiv = doc.selectFirst(selectState)
val state = stateDiv?.let {
when (it.text()) {
in ongoing -> MangaState.ONGOING
in finished -> MangaState.FINISHED
else -> null
}
}
val alt = doc.body().selectFirst(selectAlt)?.text()?.replace("Other name", "")
val auth = doc.body().selectFirst(selectAut)?.text()
manga.copy(
tags = doc.body().select(selectTag).mapNotNullToSet { a ->
MangaTag(
key = a.attr("href").substringAfter("manga-list-genre-").substringBeforeLast(".html"),
title = a.text().toTitleCase(),
source = source,
)
},
description = desc,
altTitle = alt,
author = auth,
state = state,
chapters = chaptersDeferred.await(),
)
}
override suspend fun getChapters(doc: Document): List<MangaChapter> {
val dateFormat = SimpleDateFormat(datePattern, sourceLocale)
return doc.body().select(selectChapter).mapChapters(reversed = true) { i, a ->
val href = a.attrAsRelativeUrl("href")
val dateText = a.selectFirst(selectDate)?.text()?.substringAfter("- ")
MangaChapter(
id = generateUid(href),
name = a.selectFirstOrThrow("div.chapter-name").text(),
number = i + 1,
url = href,
uploadDate = parseChapterDate(
dateFormat,
dateText,
),
source = source,
scanlator = null,
branch = null,
)
}
}
}

View File

@@ -0,0 +1,89 @@
package org.koitharu.kotatsu.parsers.site.fmreader.es
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.site.fmreader.FmreaderParser
import org.koitharu.kotatsu.parsers.util.*
@MangaSourceParser("OLIMPOSCANS", "OlimpoScans", "es")
internal class OlimpoScans(context: MangaLoaderContext) :
FmreaderParser(context, MangaSource.OLIMPOSCANS, "olimposcans.com") {
override val selectState = "ul.manga-info li:contains(Estado) a"
override val selectAlt = "ul.manga-info li:contains(Otros nombres)"
override val selectTag = "ul.manga-info li:contains(Género) a"
override val tagPrefix = "lista-de-comics-genero-"
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val tag = tags.oneOrThrowIfMany()
val url = buildString {
append("https://")
append(domain)
append(listUrl)
append("?page=")
append(page.toString())
when {
!query.isNullOrEmpty() -> {
append("&name=")
append(query.urlEncoded())
}
!tags.isNullOrEmpty() -> {
append("&genre=")
append(tag?.key.orEmpty())
}
}
append("&sort=")
when (sortOrder) {
SortOrder.POPULARITY -> append("views")
SortOrder.UPDATED -> append("last_update")
SortOrder.ALPHABETICAL -> append("name")
else -> append("last_update")
}
}
val doc = webClient.httpGet(url).parseHtml()
val lastPage =
doc.selectLast(".pagination a")?.attr("href")?.substringAfterLast("page=")?.substringBeforeLast("&artist")
?.toInt() ?: 1
if (lastPage < page) {
return emptyList()
}
return doc.select("div.thumb-item-flow").map { div ->
val href = "/" + div.selectFirstOrThrow("div.series-title a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(div.host ?: domain),
coverUrl = div.selectFirstOrThrow("div.img-in-ratio").attr("data-bg").toAbsoluteUrl(domain),
title = div.selectFirstOrThrow("div.series-title").text().orEmpty(),
altTitle = null,
rating = RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = null,
source = source,
isNsfw = isNsfwSource,
)
}
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = ("/" + chapter.url).toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
return doc.select(selectPage).map { img ->
val url = ("/proxy.php?link=" + img.src()).toRelativeUrl(domain)
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
)
}
}
}

View File

@@ -0,0 +1,125 @@
package org.koitharu.kotatsu.parsers.site.fmreader.ja
import org.jsoup.nodes.Document
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.Manga
import org.koitharu.kotatsu.parsers.model.MangaChapter
import org.koitharu.kotatsu.parsers.model.MangaPage
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.model.MangaTag
import org.koitharu.kotatsu.parsers.model.RATING_UNKNOWN
import org.koitharu.kotatsu.parsers.model.SortOrder
import org.koitharu.kotatsu.parsers.site.fmreader.FmreaderParser
import org.koitharu.kotatsu.parsers.util.*
import java.text.SimpleDateFormat
@MangaSourceParser("KLZ9", "Klz9", "ja")
internal class Klz9(context: MangaLoaderContext) :
FmreaderParser(context, MangaSource.KLZ9, "klz9.com") {
override val selectDesc = "div.row:contains(Description)"
override val selectState = "ul.manga-info li:contains(Status) a"
override val selectAlt = "ul.manga-info li:contains(Other name (s))"
override val selectTag = "ul.manga-info li:contains(Genre(s)) a"
override val selectChapter = "tr"
override val selectDate = "td i"
override val selectPage = "img"
override val selectBodyTag = "div.panel-body a"
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val tag = tags.oneOrThrowIfMany()
val url = buildString {
append("https://")
append(domain)
append("/$listUrl")
append("?page=")
append(page.toString())
when {
!query.isNullOrEmpty() -> {
append("&name=")
append(query.urlEncoded())
}
!tags.isNullOrEmpty() -> {
append("&genre=")
append(tag?.key.orEmpty())
}
}
append("&sort=")
when (sortOrder) {
SortOrder.POPULARITY -> append("views")
SortOrder.UPDATED -> append("last_update")
SortOrder.ALPHABETICAL -> append("name")
else -> append("last_update")
}
}
val doc = webClient.httpGet(url).parseHtml()
return doc.select("div.thumb-item-flow").map { div ->
val href = "/" + div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(div.host ?: domain),
coverUrl = div.selectFirstOrThrow("div.img-in-ratio").attr("style").substringAfter("('")
.substringBeforeLast("')"),
title = div.selectFirstOrThrow("div.series-title").text().orEmpty(),
altTitle = null,
rating = RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = null,
source = source,
isNsfw = isNsfwSource,
)
}
}
override suspend fun getChapters(doc: Document): List<MangaChapter> {
val slug = doc.selectFirstOrThrow("div.h0rating").attr("slug")
val docLoad =
webClient.httpGet("https://$domain/app/manga/controllers/cont.listChapter.php?slug=$slug").parseHtml()
val dateFormat = SimpleDateFormat(datePattern, sourceLocale)
return docLoad.body().select(selectChapter).mapChapters(reversed = true) { i, a ->
val href = "/" + a.selectFirstOrThrow("a.chapter").attrAsRelativeUrl("href")
val dateText = a.selectFirst(selectDate)?.text()
MangaChapter(
id = generateUid(href),
name = a.selectFirstOrThrow("a").text(),
number = i + 1,
url = href,
uploadDate = parseChapterDate(
dateFormat,
dateText,
),
source = source,
scanlator = null,
branch = null,
)
}
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
val cid = doc.selectFirstOrThrow("#chapter").attr("value")
val docLoad = webClient.httpGet("https://$domain/app/manga/controllers/cont.listImg.php?cid=$cid").parseHtml()
return docLoad.select(selectPage).map { img ->
val url = img.src()?.toRelativeUrl(domain) ?: img.parseFailed("Image src not found")
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
)
}
}
}

View File

@@ -0,0 +1,57 @@
package org.koitharu.kotatsu.parsers.site.fmreader.ja
import org.jsoup.nodes.Document
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.MangaChapter
import org.koitharu.kotatsu.parsers.model.MangaPage
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.site.fmreader.FmreaderParser
import org.koitharu.kotatsu.parsers.util.*
import java.text.SimpleDateFormat
@MangaSourceParser("WELOVEMANGA", "WeLoveManga", "ja")
internal class WeLoveManga(context: MangaLoaderContext) :
FmreaderParser(context, MangaSource.WELOVEMANGA, "welovemanga.one") {
override suspend fun getChapters(doc: Document): List<MangaChapter> {
val mid = doc.selectFirstOrThrow("div.cmt input").attr("value")
val docLoad =
webClient.httpGet("https://$domain/app/manga/controllers/cont.Listchapter.php?mid=$mid").parseHtml()
val dateFormat = SimpleDateFormat(datePattern, sourceLocale)
return docLoad.body().select(selectChapter).mapChapters(reversed = true) { i, a ->
val href = a.selectFirstOrThrow("a").attrAsRelativeUrl("href")
val dateText = a.selectFirst(selectDate)?.text()
MangaChapter(
id = generateUid(href),
name = a.selectFirstOrThrow("a").text(),
number = i + 1,
url = href,
uploadDate = parseChapterDate(
dateFormat,
dateText,
),
source = source,
scanlator = null,
branch = null,
)
}
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
val cid = doc.selectFirstOrThrow("#chapter").attr("value")
val docLoad = webClient.httpGet("https://$domain/app/manga/controllers/cont.listImg.php?cid=$cid").parseHtml()
return docLoad.select("img").map { img ->
val url = img.src()?.toRelativeUrl(domain) ?: img.parseFailed("Image src not found")
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
)
}
}
}

View File

@@ -0,0 +1,10 @@
package org.koitharu.kotatsu.parsers.site.fmreader.ja
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.site.fmreader.FmreaderParser
@MangaSourceParser("WELOMA", "Weloma", "ja")
internal class Weloma(context: MangaLoaderContext) :
FmreaderParser(context, MangaSource.WELOMA, "weloma.art")

View File

@@ -0,0 +1,163 @@
package org.koitharu.kotatsu.parsers.site.foolslide
import kotlinx.coroutines.coroutineScope
import org.json.JSONArray
import org.jsoup.nodes.Document
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import java.text.SimpleDateFormat
import java.util.*
internal abstract class FoolSlideParser(
context: MangaLoaderContext,
source: MangaSource,
domain: String,
pageSize: Int = 25,
) : PagedMangaParser(context, source, pageSize) {
override val configKeyDomain = ConfigKey.Domain(domain)
override val availableSortOrders: Set<SortOrder> = EnumSet.of(SortOrder.ALPHABETICAL)
protected open val listUrl = "directory/"
protected open val searchUrl = "search/"
protected open val pagination = true // false if the manga list has no pages
protected open val datePattern = "yyyy.MM.dd"
init {
paginator.firstPage = 1
searchPaginator.firstPage = 1
}
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val doc = if (!query.isNullOrEmpty()) {
val url = buildString {
append("https://$domain/$searchUrl")
if (page > 1) {
return emptyList()
}
}
val q = query.urlEncoded()
webClient.httpPost(url, "search=$q").parseHtml()
} else {
val url = buildString {
append("https://$domain/$listUrl")
// For some sites that don't have enough manga and page 2 links to page 1
if (!pagination) {
if (page > 1) {
return emptyList()
}
} else {
append(page.toString())
}
}
webClient.httpGet(url).parseHtml()
}
return doc.select("div.list div.group").map { div ->
val href = div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(div.host ?: domain),
coverUrl = div.selectFirst("img")?.src().orEmpty(),// in search no img
title = div.selectFirstOrThrow(".title a").text().orEmpty(),
altTitle = null,
rating = RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = null,
source = source,
isNsfw = isNsfwSource,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> = emptySet()
protected open val selectInfo = "div.info"
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val fullUrl = manga.url.toAbsoluteUrl(domain)
val testAdultPage = webClient.httpGet(fullUrl).parseHtml()
val doc = if (testAdultPage.selectFirst("div.info form") != null) {
webClient.httpPost(fullUrl, "adult=true").parseHtml()
} else {
testAdultPage
}
val chapters = getChapters(doc)
val desc = if (doc.selectFirstOrThrow(selectInfo).html().contains("</b>")) {
doc.selectFirstOrThrow(selectInfo).text().substringAfterLast(": ")
} else {
doc.selectFirstOrThrow(selectInfo).text()
}
val author = if (doc.selectFirstOrThrow(selectInfo).html().contains("</b>")) {
doc.selectFirstOrThrow(selectInfo).text().substringAfter(": ").substringBefore("Art")
} else {
null
}
manga.copy(
coverUrl = doc.selectFirst(".thumbnail img")?.src() ?: manga.coverUrl,
description = desc,
altTitle = null,
author = author,
state = null,
chapters = chapters,
)
}
protected open val selectDate = ".meta_r"
protected open val selectChapter = "div.list div.element"
protected open suspend fun getChapters(doc: Document): List<MangaChapter> {
val dateFormat = SimpleDateFormat(datePattern, sourceLocale)
return doc.body().select(selectChapter).mapChapters(reversed = true) { i, div ->
val a = div.selectFirstOrThrow(".title a")
val href = a.attrAsRelativeUrl("href")
val dateText = div.selectFirstOrThrow(selectDate).text().substringAfter(", ")
MangaChapter(
id = generateUid(href),
name = a.text(),
number = i + 1,
url = href,
uploadDate = if (div.selectFirstOrThrow(selectDate).text().contains(", ")) {
dateFormat.tryParse(dateText)
} else {
0
},
source = source,
scanlator = null,
branch = null,
)
}
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
val script = doc.selectFirstOrThrow("script:containsData(var pages = )")
val images = JSONArray(script.data().substringAfterLast("var pages = ").substringBefore(';'))
val pages = ArrayList<MangaPage>(images.length())
for (i in 0 until images.length()) {
val pageTake = images.getJSONObject(i)
pages.add(
MangaPage(
id = generateUid(pageTake.getString("url")),
url = pageTake.getString("url"),
preview = null,
source = source,
),
)
}
return pages
}
}

View File

@@ -0,0 +1,135 @@
package org.koitharu.kotatsu.parsers.site.foolslide.en
import kotlinx.coroutines.coroutineScope
import org.jsoup.nodes.Document
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.site.foolslide.FoolSlideParser
import org.koitharu.kotatsu.parsers.util.*
@MangaSourceParser("ASSORTEDSCANS", "AssortedScans", "en")
internal class AssortedScans(context: MangaLoaderContext) :
FoolSlideParser(context, MangaSource.ASSORTEDSCANS, "assortedscans.com", 56) {
override val listUrl = "reader/"
override val pagination = false
override val selectInfo = "div.#series-info"
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val doc = if (!query.isNullOrEmpty()) {
if (page > 1) {
return emptyList()
}
val url = buildString {
append("https://")
append(domain)
append('/')
append(searchUrl)
append("?q=")
append(query.urlEncoded())
}
webClient.httpGet(url).parseHtml()
} else {
val url = buildString {
append("https://$domain/$listUrl")
// For some sites that don't have enough manga and page 2 links to page 1
if (!pagination) {
if (page > 1) {
return emptyList()
}
} else {
append(page.toString())
}
}
webClient.httpGet(url).parseHtml()
}
return doc.select("section.series, tr.result").map { div ->
val href = div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(div.host ?: domain),
coverUrl = div.selectFirst("img")?.src().orEmpty(),// in search no img
title = div.selectFirstOrThrow("a").text().orEmpty(),
altTitle = null,
rating = RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = null,
source = source,
isNsfw = isNsfwSource,
)
}
}
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val fullUrl = manga.url.toAbsoluteUrl(domain)
val testAdultPage = webClient.httpGet(fullUrl).parseHtml()
val doc = if (testAdultPage.selectFirst("div.info form") != null) {
webClient.httpPost(fullUrl, "adult=true").parseHtml()
} else {
testAdultPage
}
val chapters = getChapters(doc)
val desc = doc.getElementById("series-desc")?.selectFirst("div")?.html()
val alt = doc.getElementById("series-aliases")?.selectFirst("div.alias")?.text()
val author = doc.getElementById("series-authors")?.selectFirst("div.author")?.text()
val state = doc.getElementById("series-status")?.selectFirst("span")?.text()
manga.copy(
coverUrl = doc.selectFirst(".cover")?.src() ?: manga.coverUrl,
description = desc,
altTitle = alt,
author = author,
state = when (state) {
"Ongoing" -> MangaState.ONGOING
"Completed" -> MangaState.FINISHED
"Canceled" -> MangaState.ABANDONED
else -> null
},
chapters = chapters,
)
}
override suspend fun getChapters(doc: Document): List<MangaChapter> {
return doc.body().select("div.chapter").mapChapters(reversed = true) { i, div ->
val a = div.selectFirstOrThrow("a")
val href = a.attrAsRelativeUrl("href")
MangaChapter(
id = generateUid(href),
name = a.text(),
number = i + 1,
url = href,
uploadDate = 0,
source = source,
scanlator = null,
branch = null,
)
}
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val doc = webClient.httpGet(chapter.url.toAbsoluteUrl(domain)).parseHtml()
return doc.body().select(".page-list .dropdown-list li a").map { a ->
val url = a.attr("href").toRelativeUrl(domain)
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
)
}
}
override suspend fun getPageUrl(page: MangaPage): String {
val doc = webClient.httpGet(page.url.toAbsoluteUrl(domain)).parseHtml()
val root = doc.body()
return root.requireElementById("page-image").attr("src") ?: doc.parseFailed("Page image not found")
}
}

View File

@@ -0,0 +1,10 @@
package org.koitharu.kotatsu.parsers.site.foolslide.en
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.site.foolslide.FoolSlideParser
@MangaSourceParser("DEATHTOLLSCANS", "DeathTollScans", "en")
internal class Deathtollscans(context: MangaLoaderContext) :
FoolSlideParser(context, MangaSource.DEATHTOLLSCANS, "reader.deathtollscans.net", 26)

View File

@@ -0,0 +1,12 @@
package org.koitharu.kotatsu.parsers.site.foolslide.en
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.site.foolslide.FoolSlideParser
@MangaSourceParser("MANGATELLERS", "Mangatellers", "en")
internal class Mangatellers(context: MangaLoaderContext) :
FoolSlideParser(context, MangaSource.MANGATELLERS, "reader.mangatellers.gr") {
override val pagination = false
}

View File

@@ -0,0 +1,10 @@
package org.koitharu.kotatsu.parsers.site.foolslide.en
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.site.foolslide.FoolSlideParser
@MangaSourceParser("READER_EVILFLOWERS", "Evil Flowers", "en")
internal class ReaderEvilflowers(context: MangaLoaderContext) :
FoolSlideParser(context, MangaSource.READER_EVILFLOWERS, "reader.evilflowers.com")

View File

@@ -0,0 +1,44 @@
package org.koitharu.kotatsu.parsers.site.foolslide.en
import kotlinx.coroutines.coroutineScope
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.site.foolslide.FoolSlideParser
import org.koitharu.kotatsu.parsers.util.*
@MangaSourceParser("SEINAGI", "Seinagi", "en")
internal class Seinagi(context: MangaLoaderContext) :
FoolSlideParser(context, MangaSource.SEINAGI, "reader.seinagi.org.es") {
override val pagination = false
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val fullUrl = manga.url.toAbsoluteUrl(domain)
val testAdultPage = webClient.httpGet(fullUrl).parseHtml()
val doc = if (testAdultPage.selectFirst("div.info form") != null) {
webClient.httpPost(fullUrl, "adult=true").parseHtml()
} else {
testAdultPage
}
val chapters = getChapters(doc)
val desc = if (doc.selectFirstOrThrow(selectInfo).html().contains("Description")) {
doc.selectFirstOrThrow(selectInfo).text().substringAfter("Description: ").substringBefore("Readings")
} else {
doc.selectFirstOrThrow(selectInfo).text()
}
val author = if (doc.selectFirstOrThrow(selectInfo).html().contains("Author")) {
doc.selectFirstOrThrow(selectInfo).text().substringAfter("Author: ").substringBefore("Art")
} else {
null
}
manga.copy(
coverUrl = doc.selectFirst(".thumbnail img")?.src() ?: manga.coverUrl,
description = desc,
altTitle = null,
author = author,
state = null,
chapters = chapters,
)
}
}

View File

@@ -0,0 +1,13 @@
package org.koitharu.kotatsu.parsers.site.foolslide.es
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.site.foolslide.FoolSlideParser
@MangaSourceParser("MENUDO_FANSUB", "Menudo Fansub", "es")
internal class MenudoFansub(context: MangaLoaderContext) :
FoolSlideParser(context, MangaSource.MENUDO_FANSUB, "www.menudo-fansub.com", 25) {
override val searchUrl = "slide/search/"
override val listUrl = "slide/directory/"
}

View File

@@ -0,0 +1,42 @@
package org.koitharu.kotatsu.parsers.site.foolslide.es
import kotlinx.coroutines.coroutineScope
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.site.foolslide.FoolSlideParser
import org.koitharu.kotatsu.parsers.util.*
@MangaSourceParser("PZYKOSIS666HFANSUB", "Pzykosis666h Fansub", "es", ContentType.HENTAI)
internal class Pzykosis666hFansub(context: MangaLoaderContext) :
FoolSlideParser(context, MangaSource.PZYKOSIS666HFANSUB, "lector.pzykosis666hfansub.com") {
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val fullUrl = manga.url.toAbsoluteUrl(domain)
val testAdultPage = webClient.httpGet(fullUrl).parseHtml()
val doc = if (testAdultPage.selectFirst("div.info form") != null) {
webClient.httpPost(fullUrl, "adult=true").parseHtml()
} else {
testAdultPage
}
val chapters = getChapters(doc)
val desc = if (doc.selectFirstOrThrow(selectInfo).html().contains("Descripción")) {
doc.selectFirstOrThrow(selectInfo).text().substringAfter("Descripción: ").substringBefore("Lecturas")
} else {
doc.selectFirstOrThrow(selectInfo).text()
}
val author = if (doc.selectFirstOrThrow(selectInfo).html().contains("Author")) {
doc.selectFirstOrThrow(selectInfo).text().substringAfter("Author: ").substringBefore("Art")
} else {
null
}
manga.copy(
coverUrl = doc.selectFirst(".thumbnail img")?.src() ?: manga.coverUrl,
description = desc,
altTitle = null,
author = author,
state = null,
chapters = chapters,
)
}
}

View File

@@ -0,0 +1,48 @@
package org.koitharu.kotatsu.parsers.site.foolslide.es
import kotlinx.coroutines.coroutineScope
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.ContentType
import org.koitharu.kotatsu.parsers.model.Manga
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.site.foolslide.FoolSlideParser
import org.koitharu.kotatsu.parsers.util.domain
import org.koitharu.kotatsu.parsers.util.parseHtml
import org.koitharu.kotatsu.parsers.util.selectFirstOrThrow
import org.koitharu.kotatsu.parsers.util.src
import org.koitharu.kotatsu.parsers.util.toAbsoluteUrl
@MangaSourceParser("SEINAGIADULTO", "Seinagi Adulto", "es", ContentType.HENTAI)
internal class SeinagiAdulto(context: MangaLoaderContext) :
FoolSlideParser(context, MangaSource.SEINAGIADULTO, "adulto.seinagi.org.es") {
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val fullUrl = manga.url.toAbsoluteUrl(domain)
val testAdultPage = webClient.httpGet(fullUrl).parseHtml()
val doc = if (testAdultPage.selectFirst("div.info form") != null) {
webClient.httpPost(fullUrl, "adult=true").parseHtml()
} else {
testAdultPage
}
val chapters = getChapters(doc)
val desc = if (doc.selectFirstOrThrow(selectInfo).html().contains("Descripción")) {
doc.selectFirstOrThrow(selectInfo).text().substringAfter("Descripción: ").substringBefore("Lecturas")
} else {
doc.selectFirstOrThrow(selectInfo).text()
}
val author = if (doc.selectFirstOrThrow(selectInfo).html().contains("Author")) {
doc.selectFirstOrThrow(selectInfo).text().substringAfter("Author: ").substringBefore("Art")
} else {
null
}
manga.copy(
coverUrl = doc.selectFirst(".thumbnail img")?.src().orEmpty(),// for manga result on search
description = desc,
altTitle = null,
author = author,
state = null,
chapters = chapters,
)
}
}

View File

@@ -0,0 +1,14 @@
package org.koitharu.kotatsu.parsers.site.foolslide.fr
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.site.foolslide.FoolSlideParser
@MangaSourceParser("HNISCANTRAD", "Hni Scantrad", "fr")
internal class HniScantrad(context: MangaLoaderContext) :
FoolSlideParser(context, MangaSource.HNISCANTRAD, "hni-scantrad.com") {
override val pagination = false
override val searchUrl = "lel/search/"
override val listUrl = "lel/directory/"
}

View File

@@ -0,0 +1,14 @@
package org.koitharu.kotatsu.parsers.site.foolslide.it
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.site.foolslide.FoolSlideParser
@MangaSourceParser("POWERMANGA", "PowerManga", "it")
internal class PowerManga(context: MangaLoaderContext) :
FoolSlideParser(context, MangaSource.POWERMANGA, "reader.powermanga.org") {
override val pagination = false
}

View File

@@ -0,0 +1,15 @@
package org.koitharu.kotatsu.parsers.site.foolslide.it
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.site.foolslide.FoolSlideParser
@MangaSourceParser("RAMAREADER", "RamaReader", "it")
internal class Ramareader(context: MangaLoaderContext) :
FoolSlideParser(context, MangaSource.RAMAREADER, "www.ramareader.it") {
override val searchUrl = "read/search/"
override val listUrl = "read/directory/"
}

View File

@@ -0,0 +1,15 @@
package org.koitharu.kotatsu.parsers.site.foolslide.it
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.model.MangaSource
import org.koitharu.kotatsu.parsers.site.foolslide.FoolSlideParser
@MangaSourceParser("READNIFTEAM", "ReadNifTeam", "it")
internal class ReadNifteam(context: MangaLoaderContext) :
FoolSlideParser(context, MangaSource.READNIFTEAM, "read-nifteam.info") {
override val searchUrl = "slide/search/"
override val listUrl = "slide/directory/"
}

View File

@@ -0,0 +1,229 @@
package org.koitharu.kotatsu.parsers.site.fr
import kotlinx.coroutines.async
import kotlinx.coroutines.awaitAll
import kotlinx.coroutines.coroutineScope
import okhttp3.Headers
import org.jsoup.nodes.Element
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.exception.ParseException
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import org.koitharu.kotatsu.parsers.util.json.getIntOrDefault
import java.util.*
@MangaSourceParser("BENTOMANGA", "BentoManga", "fr")
internal class BentomangaParser(context: MangaLoaderContext) : PagedMangaParser(context, MangaSource.BENTOMANGA, 10) {
override val availableSortOrders: Set<SortOrder> = EnumSet.of(
SortOrder.UPDATED,
SortOrder.POPULARITY,
SortOrder.RATING,
SortOrder.NEWEST,
SortOrder.ALPHABETICAL,
)
override val configKeyDomain = ConfigKey.Domain("bentomanga.com", "www.bentomanga.com")
init {
paginator.firstPage = 0
searchPaginator.firstPage = 0
}
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val url = urlBuilder()
.addPathSegment("manga_list")
.addQueryParameter("limit", page.toString())
.addQueryParameter(
"order_by",
when (sortOrder) {
SortOrder.UPDATED -> "update"
SortOrder.POPULARITY -> "views"
SortOrder.RATING -> "top"
SortOrder.NEWEST -> "create"
SortOrder.ALPHABETICAL -> "name"
},
)
if (!tags.isNullOrEmpty()) {
url.addQueryParameter("withCategories", tags.joinToString(",") { it.key })
}
if (!query.isNullOrEmpty()) {
url.addQueryParameter("search", query)
}
val root = webClient.httpGet(url.build()).parseHtml().requireElementById("mangas_content")
return root.select(".manga[data-manga]").map { div ->
val header = div.selectFirstOrThrow(".manga_header")
val href = header.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
title = div.selectFirstOrThrow("h1").text(),
altTitle = null,
url = href,
publicUrl = href.toAbsoluteUrl(domain),
rating = div.getElementsByAttributeValue("data-icon", "avg_rate")
.firstOrNull()
?.textOrNull()
?.toFloatOrNull()
?.div(10f)
?: RATING_UNKNOWN,
isNsfw = div.selectFirst(".badge-adult_content") != null,
coverUrl = div.selectFirstOrThrow("img").src().assertNotNull("src").orEmpty(),
tags = div.selectFirst(".component-manga-categories")
.assertNotNull("tags")
?.select("a")
?.mapToSet { a ->
MangaTag(
title = a.text().toTitleCase(sourceLocale),
key = a.attr("href").substringAfterLast('='),
source = source,
)
}.orEmpty(),
state = null,
author = null,
description = div.selectFirst(".manga_synopsis")?.html().assertNotNull("description"),
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga {
val mangaUrl = manga.url.toAbsoluteUrl(domain)
val root = webClient.httpGet(mangaUrl).parseHtml()
.requireElementById("container_manga_show")
return manga.copy(
altTitle = root.selectFirst(".component-manga-title_alt")?.text(),
description = root.selectFirst(".datas_synopsis")?.html().assertNotNull("description")
?: manga.description,
state = when (root.selectFirst(".datas_more-status-data")?.textOrNull().assertNotNull("status")) {
"En cours" -> MangaState.ONGOING
"Terminé" -> MangaState.FINISHED
"Abandonné" -> MangaState.ABANDONED
"En pause" -> MangaState.PAUSED
else -> null
},
author = root.selectFirst(".datas_more-authors-people")?.textOrNull().assertNotNull("author"),
chapters = run {
val input = root.selectFirst("input[name=\"limit\"]") ?: return@run parseChapters(root)
val max = input.attr("max").toInt()
if (max <= 1) {
parseChapters(root)
} else {
coroutineScope {
val result = ArrayList(parseChapters(root))
result.ensureCapacity(result.size * max)
(1..max).map { i ->
async {
loadChapters(mangaUrl, i)
}
}.awaitAll()
.flattenTo(result)
result
}
}
}.reversed(),
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val chapterUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(chapterUrl).parseHtml()
val chapterId = doc.head().getElementsByAttribute("data-chapter-id").first()!!.attr("data-chapter-id")
val json = webClient.httpGet(
"https://$domain/api/?id=$chapterId&type=chapter",
Headers.headersOf(
"Referer", chapterUrl,
"x-requested-with", "XMLHttpRequest",
),
).parseJson()
if (json.getIntOrDefault("type", 1) == 2) {
throw ParseException("Light Novels are not supported", chapterUrl)
}
val baseUrl = json.getString("baseImagesUrl")
val pages = json.getJSONArray("page_array")
return (0 until pages.length()).map { i ->
val url = concatUrl(baseUrl, pages.getString(i))
MangaPage(
id = generateUid(url),
url = url.toAbsoluteUrl(domain),
preview = null,
source = source,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
val root = webClient.httpGet(urlBuilder().addPathSegment("manga_list").build())
.parseHtml()
.requireElementById("search_options-form")
return root.getElementsByAttributeValue("name", "categories[]")
.mapToSet { input ->
val div = input.parents().first()!!
MangaTag(
title = div.text().toTitleCase(sourceLocale),
key = input.attr("value"),
source = source,
)
}
}
private suspend fun loadChapters(baseUrl: String, page: Int): List<MangaChapter> {
return parseChapters(webClient.httpGet("$baseUrl?limit=$page").parseHtml().body())
}
private fun parseChapters(root: Element): List<MangaChapter> {
return root.requireElementById("chapters_content")
.select(".component-chapter").map { div ->
val a = div.selectFirstOrThrow("a:not([style*='display:none'])")
val href = a.attrAsRelativeUrl("href")
val title = div.selectFirstOrThrow(".chapter_volume").text()
val name = div.selectFirst(".chapter_title")?.textOrNull()
MangaChapter(
id = generateUid(href),
name = if (name != null && name != title) "$title: $name" else title,
number = href.substringAfterLast('/').toIntOrNull() ?: 0,
url = href,
scanlator = div.selectFirst(".team_link-name")?.textOrNull(),
uploadDate = div.selectFirst(".component-chapter-date")
?.ownTextOrNull()
.parseDate(),
branch = null,
source = source,
)
}
}
private fun String?.parseDate(): Long {
if (this == null) {
assert(false) { "Date is null" }
return 0L
}
val parts = split(' ')
assert(parts.size == 2) { "Wrong date $this" }
val count = parts.getOrNull(0)?.toIntOrNull() ?: return 0L
val unit = parts.getOrNull(1) ?: return 0L
val calendarUnit = when (unit) {
"s" -> Calendar.SECOND
"min" -> Calendar.MINUTE
"h" -> Calendar.HOUR
"j" -> Calendar.DAY_OF_YEAR
"sem." -> Calendar.WEEK_OF_YEAR
"mois" -> Calendar.MONTH
"ans", "an" -> Calendar.YEAR
else -> {
assert(false) { "Unknown time unit $unit" }
return 0L
}
}
val calendar = Calendar.getInstance()
calendar.add(calendarUnit, -count)
return calendar.timeInMillis
}
}

View File

@@ -0,0 +1,148 @@
package org.koitharu.kotatsu.parsers.site.fr
import kotlinx.coroutines.coroutineScope
import org.json.JSONArray
import org.json.JSONObject
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.util.*
import org.koitharu.kotatsu.parsers.util.json.*
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("FMTEAM", "FmTeam", "fr")
internal class FmTeam(context: MangaLoaderContext) :
PagedMangaParser(context, MangaSource.FMTEAM, 0) {
override val availableSortOrders: Set<SortOrder> = EnumSet.of(SortOrder.ALPHABETICAL)
override val configKeyDomain = ConfigKey.Domain("fmteam.fr")
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
if (page > 1) {
return emptyList()
}
val jsonManga = if (!query.isNullOrEmpty()) {
//3 letters minimum
webClient.httpGet("https://$domain/api/search/${query.urlEncoded()}").parseJson().getJSONArray("comics")
} else {
webClient.httpGet("https://$domain/api/comics").parseJson().getJSONArray("comics")
}
val manga = ArrayList<Manga>(jsonManga.length())
for (i in 0 until jsonManga.length()) {
val j = jsonManga.getJSONObject(i)
val href = "/api" + j.getString("url")
when {
!tags.isNullOrEmpty() -> {
val a = j.getJSONArray("genres").toString()
var found = true
tags.forEach {
if (!a.contains(it.key, ignoreCase = true)) {
found = false
}
}
if (found) {
manga.add(
addManga(href, j),
)
}
}
else -> {
manga.add(
addManga(href, j),
)
}
}
}
return manga
}
private fun addManga(href: String, j: JSONObject): Manga {
return Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(domain),
coverUrl = j.getString("thumbnail"),
title = j.getString("title"),
description = j.getString("description"),
altTitle = j.getJSONArray("alt_titles").toString()
.replace("[\"", "")
.replace("\"]", "")
.replace("\",\"", " , "),
rating = j.getString("rating").toFloatOrNull()?.div(10f)
?: RATING_UNKNOWN,
tags = emptySet(),
author = j.getString("author"),
state = when (j.getString("status").lowercase()) {
"en cours" -> MangaState.ONGOING
"terminé" -> MangaState.FINISHED
else -> null
},
source = source,
isNsfw = when (j.getString("adult").toInt()) {
0 -> false
1 -> true
else -> true
},
)
}
override suspend fun getAvailableTags(): Set<MangaTag> = emptySet()
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val fullUrl = manga.url.toAbsoluteUrl(domain)
val json = webClient.httpGet(fullUrl).parseJson().getJSONObject("comic")
val dateFormat = SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'", Locale.US)
val chapters = JSONArray(json.getJSONArray("chapters").toJSONList().reversed())
manga.copy(
tags = json.getJSONArray("genres").toJSONList().mapNotNullToSet {
MangaTag(
key = it.getString("slug"),
title = it.getString("name"),
source = source,
)
},
chapters = chapters.mapJSONIndexed { i, j ->
val url = "/api" + j.getString("url").toRelativeUrl(domain)
val name = j.getString("full_title")
val date = j.getStringOrNull("updated_at")
MangaChapter(
id = generateUid(url),
name = name,
number = i + 1,
url = url,
scanlator = null,
uploadDate = dateFormat.tryParse(date),
branch = null,
source = source,
)
},
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val jsonPages = webClient.httpGet(fullUrl).parseJson().getJSONObject("chapter").getJSONArray("pages").toString()
val pages = jsonPages.replace("[", "").replace("]", "")
.replace("\\", "").split("\",\"").drop(1)
return pages.map { url ->
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
)
}
}
}

View File

@@ -0,0 +1,158 @@
package org.koitharu.kotatsu.parsers.site.fr
import kotlinx.coroutines.coroutineScope
import okhttp3.Headers
import org.jsoup.nodes.Document
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.network.UserAgents
import org.koitharu.kotatsu.parsers.util.*
import java.text.DateFormat
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("FURYOSOCIETY", "FuryoSociety", "fr")
internal class FuryoSociety(context: MangaLoaderContext) :
PagedMangaParser(context, MangaSource.FURYOSOCIETY, 0) {
override val availableSortOrders: Set<SortOrder> = EnumSet.of(SortOrder.ALPHABETICAL, SortOrder.UPDATED)
override val configKeyDomain = ConfigKey.Domain("furyosociety.com")
override val headers: Headers = Headers.Builder()
.add("User-Agent", UserAgents.CHROME_DESKTOP)
.build()
override suspend fun getFavicons(): Favicons {
return Favicons(
listOf(
Favicon("https://$domain/fs_favicon/favicon-32x32.png", 32, null),
),
domain,
)
}
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val url = buildString {
append("https://")
append(domain)
if (page == 1) {
if (sortOrder == SortOrder.ALPHABETICAL) {
append("/mangas")
}
} else {
return emptyList()
}
}
val doc = webClient.httpGet(url).parseHtml()
return doc.select("div.fs-card-container div.grid-item-container").ifEmpty {
doc.select("div.container-tight.latest table tr")
}.map { div ->
val href = div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(div.host ?: domain),
coverUrl = div.selectFirst("img")?.src().orEmpty(),
title = (div.selectFirst("div.media-body") ?: div.selectFirst("a"))?.text().orEmpty(),
altTitle = null,
rating = RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = null,
source = source,
isNsfw = false,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> = emptySet()
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val fullUrl = manga.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
val chaptersDeferred = getChapters(doc)
manga.copy(
description = doc.selectFirstOrThrow("div.fs-comic-description").html(),
chapters = chaptersDeferred,
isNsfw = doc.selectFirst(".adult-text") != null,
)
}
private fun getChapters(doc: Document): List<MangaChapter> {
return doc.body().select("div.list.fs-chapter-list div.element").mapChapters(reversed = true) { i, div ->
val a = div.selectFirstOrThrow("div.title a")
val href = a.attrAsRelativeUrl("href")
val dateFormat = SimpleDateFormat("dd/MM/yyyy", sourceLocale)
val dateText = div.selectFirstOrThrow("div.meta_r").text().replace("Hier", "1 jour")
MangaChapter(
id = generateUid(href),
name = div.selectFirstOrThrow("div.title").text() + " : " + div.selectFirstOrThrow("div.name").text(),
number = i + 1,
url = href,
uploadDate = parseChapterDate(
dateFormat,
dateText,
),
source = source,
scanlator = null,
branch = null,
)
}
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
return doc.select("div.main-img img").map { url ->
val img = url.src()?.toRelativeUrl(domain) ?: url.parseFailed("Image src not found")
MangaPage(
id = generateUid(img),
url = img,
preview = null,
source = source,
)
}
}
private fun parseChapterDate(dateFormat: DateFormat, date: String?): Long {
val d = date?.lowercase() ?: return 0
return when {
d.startsWith("il y a") || // Handle translated 'ago' in French.
d.endsWith(" an") || d.endsWith(" ans") ||
d.endsWith(" mois") ||
d.endsWith(" jour") || d.endsWith(" jours") ||
d.endsWith(" heure") || d.endsWith(" heures") ||
d.endsWith(" seconde") || d.endsWith(" secondes") ||
d.endsWith(" minute") || d.endsWith(" minutes") -> parseRelativeDate(date)
else -> dateFormat.tryParse(date)
}
}
private fun parseRelativeDate(date: String): Long {
val number = Regex("""(\d+)""").find(date)?.value?.toIntOrNull() ?: return 0
val cal = Calendar.getInstance()
return when {
WordSet("seconde", "secondes").anyWordIn(date) -> cal.apply { add(Calendar.SECOND, -number) }.timeInMillis
WordSet("minute", "minutes").anyWordIn(date) -> cal.apply { add(Calendar.MINUTE, -number) }.timeInMillis
WordSet("heure", "heures").anyWordIn(date) -> cal.apply { add(Calendar.HOUR, -number) }.timeInMillis
WordSet("jour", "jours").anyWordIn(date) -> cal.apply { add(Calendar.DAY_OF_MONTH, -number) }.timeInMillis
WordSet("mois").anyWordIn(date) -> cal.apply { add(Calendar.MONTH, -number) }.timeInMillis
WordSet("an", "ans").anyWordIn(date) -> cal.apply { add(Calendar.YEAR, -number) }.timeInMillis
else -> 0
}
}
}

View File

@@ -0,0 +1,168 @@
package org.koitharu.kotatsu.parsers.site.fr
import okhttp3.Headers
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.network.UserAgents
import org.koitharu.kotatsu.parsers.util.*
import org.koitharu.kotatsu.parsers.util.json.mapJSON
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("LEGACY_SCANS", "LegacyScans", "fr")
internal class LegacyScansParser(context: MangaLoaderContext) :
PagedMangaParser(context, MangaSource.LEGACY_SCANS, 18) {
override val availableSortOrders: Set<SortOrder> = EnumSet.of(
SortOrder.POPULARITY,
)
override val configKeyDomain = ConfigKey.Domain("legacy-scans.com")
override val headers: Headers = Headers.Builder()
.add("User-Agent", UserAgents.CHROME_MOBILE)
.build()
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val end = page * pageSize
val start = end - (pageSize - 1)
val url = if (!query.isNullOrEmpty()) {
if (page > 1) {
return emptyList()
}
buildString {
append("https://api.$domain/misc/home/search?title=")
append(query.urlEncoded())
}
} else {
buildString {
append("https://api.$domain/misc/comic/search/query?status=&order=&genreNames=")
if (!tags.isNullOrEmpty()) {
for (tag in tags) {
append(tag.key)
append(',')
}
}
append("&type=&start=")
append(start)
append("&end=")
append(end)
}
}
val json = webClient.httpGet(url).parseJson()
return if (!query.isNullOrEmpty()) {
json.getJSONArray("results").mapJSON { j ->
val slug = j.getString("slug")
val urlManga = "https://$domain/comics/$slug"
Manga(
id = generateUid(urlManga),
title = j.getString("title"),
altTitle = null,
url = urlManga,
publicUrl = urlManga,
rating = RATING_UNKNOWN,
isNsfw = false,
coverUrl = "",
tags = setOf(),
state = null,
author = null,
source = source,
)
}
} else {
json.getJSONArray("comics").mapJSON { j ->
val slug = j.getString("slug")
val urlManga = "https://$domain/comics/$slug"
Manga(
id = generateUid(urlManga),
title = j.getString("title"),
altTitle = null,
url = urlManga,
publicUrl = urlManga,
rating = RATING_UNKNOWN,
isNsfw = false,
coverUrl = "https://api.$domain/" + j.getString("cover"),
tags = setOf(),
state = null,
author = null,
source = source,
)
}
}
}
override suspend fun getDetails(manga: Manga): Manga {
val root = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
val dateFormat = SimpleDateFormat("dd/MM/yyyy", Locale.FRENCH)
return manga.copy(
altTitle = null,
tags = root.select("div.serieGenre span").mapNotNullToSet { span ->
MangaTag(
key = span.text(),
title = span.text(),
source = source,
)
},
coverUrl = root.selectFirstOrThrow("div.serieImg img").attr("src"),
author = root.select("div.serieAdd p:contains(Auteur:) strong").text(),
description = root.selectFirst("div.serieDescription div")?.html(),
chapters = root.select("div.chapterList a")
.mapChapters(reversed = true) { i, a ->
val href = a.attrAsRelativeUrl("href")
val name = a.selectFirstOrThrow("span").text()
val dateText = a.selectLast("span")?.text() ?: "0"
MangaChapter(
id = generateUid(href),
name = name,
number = i + 1,
url = href,
scanlator = null,
uploadDate = dateFormat.tryParse(dateText),
branch = null,
source = source,
)
},
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
return doc.select("div.readerComics img").map { img ->
val url = img.src() ?: img.parseFailed("Image src not found")
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
val doc = webClient.httpGet("https://$domain/comics").parseHtml()
val script = doc.requireElementById("__NUXT_DATA__").data()
.substringAfterLast("\"genres\"").substringBeforeLast("\"comics\"")
.split("\",\"").drop(1)
return script.mapNotNullToSet { tag ->
MangaTag(
key = tag.substringBeforeLast("\",{"),
title = tag.substringBeforeLast("\",{"),
source = source,
)
}
}
}

View File

@@ -0,0 +1,146 @@
package org.koitharu.kotatsu.parsers.site.fr
import okhttp3.Headers
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.network.UserAgents
import org.koitharu.kotatsu.parsers.util.*
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("LIRESCAN", "LireScan", "fr")
internal class LireScan(context: MangaLoaderContext) : PagedMangaParser(context, MangaSource.LIRESCAN, 20) {
override val availableSortOrders: Set<SortOrder> = EnumSet.of(SortOrder.UPDATED)
override val configKeyDomain = ConfigKey.Domain("lire-scan.me")
override val isMultipleTagsSupported = false
override val headers: Headers = Headers.Builder()
.add("User-Agent", UserAgents.CHROME_MOBILE)
.build()
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val tag = tags.oneOrThrowIfMany()
val doc =
if (!query.isNullOrEmpty()) { // search only works with 4 or more letters
if (page > 1) {
return emptyList()
}
val q = query.urlEncoded().replace("%20", "+")
val post = "do=search&subaction=search&search_start=0&full_search=0&result_from=1&story=$q"
webClient.httpPost("https://$domain/index.php?do=search", post).parseHtml()
} else {
val url = buildString {
append("https://")
append(domain)
if (!tags.isNullOrEmpty()) {
append("/manga/")
append(tag?.key.orEmpty())
}
if (page > 1) {
append("/page/")
append(page)
append('/')
}
}
webClient.httpGet(url).parseHtml()
}
return doc.select("div.sect__content.grid-items div.item-poster").map { div ->
val href = div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
title = div.select(".item-poster__title").text(),
altTitle = null,
url = href,
publicUrl = href.toAbsoluteUrl(domain),
rating = div.selectFirstOrThrow(".item__rating").ownText().toFloatOrNull()?.div(10f) ?: RATING_UNKNOWN,
isNsfw = false,
coverUrl = div.selectFirstOrThrow("img").attrAsAbsoluteUrl("src"),
tags = setOf(),
state = null,
author = null,
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga {
val root = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml()
val dateFormat = SimpleDateFormat("dd-MM-yyyy", Locale.FRANCE)
return manga.copy(
altTitle = root.select("ul.pmovie__list li:contains(Nom Alternatif:)").text()
.replace("Nom Alternatif:", ""),
state = when (root.select("ul.pmovie__list li:contains(Status:)").text()) {
"Status: OnGoing", "Status: En cours" -> MangaState.ONGOING
"Status: Fini" -> MangaState.FINISHED
else -> null
},
tags = root.select("ul.pmovie__list li:contains(Genre:)").text()
.replace("Genre:", "").split(" / ").mapNotNullToSet { tag ->
MangaTag(
key = tag.lowercase(),
title = tag,
source = source,
)
},
author = root.select("ul.pmovie__list li:contains(Artist(s):)").text().replace("Artist(s):", ""),
description = root.selectFirst("div.pmovie__text")?.html(),
chapters = root.select("ul li div.chapter")
.mapChapters(reversed = true) { i, div ->
val a = div.selectFirstOrThrow("a")
val href = a.attrAsRelativeUrl("href")
val name = a.text()
val dateText = div.select("p").last()?.text()
MangaChapter(
id = generateUid(href),
name = name,
number = i,
url = href,
scanlator = null,
uploadDate = dateFormat.tryParse(dateText),
branch = null,
source = source,
)
},
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
val pages = doc.selectFirstOrThrow("script:containsData(const manga = )").data()
.substringAfter("chapter1: [\"").substringBefore("\"]")
.split("\",\"")
return pages.map { img ->
MangaPage(
id = generateUid(img),
url = img,
preview = null,
source = source,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
val doc = webClient.httpGet("https://$domain/").parseHtml()
return doc.select(".nav-menu li a").mapNotNullToSet { a ->
val key = a.attr("href").removeSuffix('/').substringAfterLast("manga/", "")
MangaTag(
key = key,
title = a.text(),
source = source,
)
}
}
}

View File

@@ -0,0 +1,181 @@
package org.koitharu.kotatsu.parsers.site.fr
import okhttp3.Headers
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.network.UserAgents
import org.koitharu.kotatsu.parsers.util.*
import org.koitharu.kotatsu.parsers.util.json.mapJSON
import java.lang.IllegalArgumentException
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("LUGNICASCANS", "LugnicaScans", "fr")
internal class LugnicaScans(context: MangaLoaderContext) : PagedMangaParser(context, MangaSource.LUGNICASCANS, 10) {
override val availableSortOrders: Set<SortOrder> = EnumSet.of(
SortOrder.ALPHABETICAL,
SortOrder.UPDATED,
)
override val configKeyDomain = ConfigKey.Domain("lugnica-scans.com")
override val headers: Headers = Headers.Builder()
.add("User-Agent", UserAgents.CHROME_DESKTOP)
.build()
init {
context.cookieJar.insertCookies(
domain,
"reader_render=continue;",
)
}
override suspend fun getFavicons(): Favicons {
return Favicons(
listOf(
Favicon("https://$domain/favicon/favicon-32x32.png", 32, null),
),
domain,
)
}
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
if (!query.isNullOrEmpty()) {
throw IllegalArgumentException("Search is not supported by this source")
}
if (sortOrder == SortOrder.ALPHABETICAL) {
if (page > 1) {
return emptyList()
}
val url = buildString {
append("https://")
append(domain)
append("/api/get/catalog?page=0&filter=all")
}
val json = webClient.httpGet(url).parseJsonArray()
return json.mapJSON { j ->
val urlManga = "https://$domain/api/get/card/${j.getString("slug")}"
val img = "https://$domain/upload/min_cover/${j.getString("image")}"
Manga(
id = generateUid(urlManga),
title = j.getString("title"),
altTitle = null,
url = urlManga,
publicUrl = urlManga.toAbsoluteUrl(domain),
rating = j.getString("rate").toFloatOrNull()?.div(5f) ?: RATING_UNKNOWN,
isNsfw = false,
coverUrl = img,
tags = setOf(),
state = when (j.getString("status")) {
"0" -> MangaState.ONGOING
"1" -> MangaState.FINISHED
"3" -> MangaState.ABANDONED
else -> null
},
author = null,
source = source,
)
}
} else {
val url = buildString {
append("https://")
append(domain)
append("/api/get/homegrid/")
append(page)
}
val json = webClient.httpGet(url).parseJsonArray()
return json.mapJSON { j ->
val urlManga = "https://$domain/api/get/card/${j.getString("manga_slug")}"
val img = "https://$domain/upload/min_cover/${j.getString("manga_image")}"
Manga(
id = generateUid(urlManga),
title = j.getString("manga_title"),
altTitle = null,
url = urlManga,
publicUrl = urlManga.toAbsoluteUrl(domain),
rating = j.getString("manga_rate").toFloatOrNull()?.div(5f) ?: RATING_UNKNOWN,
isNsfw = false,
coverUrl = img,
tags = setOf(),
state = null,
author = null,
source = source,
)
}
}
}
override suspend fun getDetails(manga: Manga): Manga {
val json = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseJson()
val jsonManga = json.getJSONObject("manga")
val chapters = json.getJSONObject("chapters").toString().split("{\"id\":").drop(1)
val slug = manga.url.substringAfterLast("/")
val dateFormat = SimpleDateFormat("dd-MM-yyyy", Locale.FRANCE)
return manga.copy(
altTitle = null,
state = when (jsonManga.getString("status")) {
"0" -> MangaState.ONGOING
"1" -> MangaState.FINISHED
"3" -> MangaState.ABANDONED
else -> null
},
author = jsonManga.getString("author"),
description = jsonManga.getString("description"),
chapters = chapters.mapChapters(reversed = true) { i, it ->
val id = it.substringAfter("\"chapter\":").substringBefore(",")
val url = "https://$domain/api/get/chapter/$slug/$id"
val date = getDateString(
it.substringAfter("\"date\":\"").substringBefore("\",").toLong(),
)
MangaChapter(
id = generateUid(url),
name = "Chapitre : $id",
number = i,
url = url,
scanlator = null,
uploadDate = dateFormat.tryParse(date),
branch = null,
source = source,
)
},
)
}
private val simpleDateFormat = SimpleDateFormat("dd-MM-yyyy", Locale.FRANCE)
private fun getDateString(time: Long): String = simpleDateFormat.format(time * 1000L)
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val jsonPage = webClient.httpGet(fullUrl).parseJson()
val idManga = jsonPage.getJSONObject("manga").getInt("id")
val slug = jsonPage.getJSONObject("chapter").getInt("chapter")
val jsonPages = jsonPage.getJSONObject("chapter").getJSONArray("files")
val pages = ArrayList<MangaPage>(jsonPages.length())
for (i in 0 until jsonPages.length()) {
val url = "https://$domain/upload/chapters/$idManga/$slug/${jsonPages.getString(i)}"
pages.add(
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
),
)
}
return pages
}
override suspend fun getAvailableTags(): Set<MangaTag> = emptySet()
}

View File

@@ -0,0 +1,183 @@
package org.koitharu.kotatsu.parsers.site.fr
import kotlinx.coroutines.coroutineScope
import okhttp3.Headers
import org.json.JSONArray
import org.jsoup.nodes.Document
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.network.UserAgents
import org.koitharu.kotatsu.parsers.util.*
import java.util.*
@MangaSourceParser("SCANS_MANGAS_ME", "ScansMangas.me", "fr")
internal class ScansMangasMe(context: MangaLoaderContext) :
PagedMangaParser(context, MangaSource.SCANS_MANGAS_ME, 0) {
override val availableSortOrders: Set<SortOrder> = EnumSet.of(
SortOrder.ALPHABETICAL,
SortOrder.UPDATED,
SortOrder.NEWEST,
SortOrder.POPULARITY,
)
override val configKeyDomain = ConfigKey.Domain("scansmangas.me")
override val headers: Headers = Headers.Builder()
.add("User-Agent", UserAgents.CHROME_DESKTOP)
.build()
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val url = buildString {
append("https://")
append(domain)
if (page == 1) {
if (!query.isNullOrEmpty()) {
append("/?s=")
append(query.urlEncoded())
append("&post_type=manga")
} else if (!tags.isNullOrEmpty()) {
append("/genres/")
for (tag in tags) {
append(tag.key)
}
} else {
append("/tous-nos-mangas/")
append("?order=")
when (sortOrder) {
SortOrder.POPULARITY -> append("popular")
SortOrder.UPDATED -> append("update")
SortOrder.ALPHABETICAL -> append("title")
SortOrder.NEWEST -> append("create")
else -> append("update")
}
}
} else {
return emptyList()
}
}
val doc = webClient.httpGet(url).parseHtml()
return doc.select("div.postbody .bs .bsx").map { div ->
val href = div.selectFirstOrThrow("a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
url = href,
publicUrl = href.toAbsoluteUrl(div.host ?: domain),
coverUrl = div.selectFirst("img")?.src().orEmpty(),
title = div.selectFirstOrThrow("div.bigor div.tt").text().orEmpty(),
altTitle = null,
rating = div.selectFirstOrThrow("div.rating i").ownText().toFloatOrNull()?.div(10f)
?: RATING_UNKNOWN,
tags = emptySet(),
author = null,
state = null,
source = source,
isNsfw = isNsfwSource,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
val doc = webClient.httpGet("https://$domain/tous-nos-mangas/").parseHtml()
return doc.select("ul.genre li").mapNotNullToSet { li ->
val key = li.selectFirstOrThrow("a").attr("href").removeSuffix('/').substringAfterLast('/')
val name = li.selectFirstOrThrow("a").text()
MangaTag(
key = key,
title = name,
source = source,
)
}
}
override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
val fullUrl = manga.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
val chaptersDeferred = getChapters(doc)
val desc = doc.selectFirstOrThrow("div.desc").html()
val state = if (doc.select("div.spe span:contains(En cours)").isNullOrEmpty()) {
MangaState.FINISHED
} else {
MangaState.ONGOING
}
val alt = doc.body().select("div.infox span.alter").text()
val aut = doc.select("div.spe span")[2].text().replace("Auteur:", "")
manga.copy(
tags = doc.select("div.spe span:contains(Genres) a").mapNotNullToSet { a ->
MangaTag(
key = a.attr("href").removeSuffix('/').substringAfterLast('/'),
title = a.text().toTitleCase(),
source = source,
)
},
description = desc,
altTitle = alt,
author = aut,
state = state,
chapters = chaptersDeferred,
isNsfw = manga.isNsfw,
)
}
private fun getChapters(doc: Document): List<MangaChapter> {
return doc.body().requireElementById("chapter_list").select("li").mapChapters(reversed = true) { i, li ->
val a = li.selectFirstOrThrow("a")
val href = a.attrAsRelativeUrl("href")
MangaChapter(
id = generateUid(href),
name = li.selectFirstOrThrow("span.mobile chapter").text(),
number = i + 1,
url = href,
uploadDate = 0,
source = source,
scanlator = null,
branch = null,
)
}
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
val script = doc.selectFirstOrThrow("script:containsData(page_image)")
val images = JSONArray(script.data().substringAfterLast("var pages = ").substringBefore(';'))
val pages = ArrayList<MangaPage>(images.length())
for (i in 0 until images.length()) {
val pageTake = images.getJSONObject(i)
pages.add(
MangaPage(
id = generateUid(pageTake.getString("page_image")),
url = pageTake.getString("page_image"),
preview = null,
source = source,
),
)
}
return pages
}
}

View File

@@ -0,0 +1,191 @@
package org.koitharu.kotatsu.parsers.site.fr
import okhttp3.Headers
import org.koitharu.kotatsu.parsers.MangaLoaderContext
import org.koitharu.kotatsu.parsers.MangaSourceParser
import org.koitharu.kotatsu.parsers.PagedMangaParser
import org.koitharu.kotatsu.parsers.config.ConfigKey
import org.koitharu.kotatsu.parsers.exception.ParseException
import org.koitharu.kotatsu.parsers.model.*
import org.koitharu.kotatsu.parsers.network.UserAgents
import org.koitharu.kotatsu.parsers.util.*
import java.text.SimpleDateFormat
import java.util.*
@MangaSourceParser("SCANTRADUNION", "ScantradUnion", "fr")
internal class ScantradUnion(context: MangaLoaderContext) : PagedMangaParser(context, MangaSource.SCANTRADUNION, 10) {
override val availableSortOrders: Set<SortOrder> = EnumSet.of(
SortOrder.ALPHABETICAL,
SortOrder.UPDATED,
)
override val configKeyDomain = ConfigKey.Domain("scantrad-union.com")
override val headers: Headers = Headers.Builder()
.add("User-Agent", UserAgents.CHROME_DESKTOP)
.build()
override suspend fun getListPage(
page: Int,
query: String?,
tags: Set<MangaTag>?,
sortOrder: SortOrder,
): List<Manga> {
val url = buildString {
append("https://")
append(domain)
when {
!query.isNullOrEmpty() -> {
append("/page/")
append(page.toString())
append("/?s=")
append(query.urlEncoded())
}
!tags.isNullOrEmpty() -> {
append("/tag/")
for (tag in tags) {
append(tag.key)
append(',')
}
append("/page/")
append(page.toString())
}
else -> {
if (sortOrder == SortOrder.ALPHABETICAL) {
append("/manga/")
append("/page/")
append(page.toString())
}
if (sortOrder == SortOrder.UPDATED) {
append("")
}
}
}
}
val doc = webClient.httpGet(url).parseHtml()
if (doc.getElementById("dernierschapitres") != null) {
val root = doc.requireElementById("dernierschapitres")
return root.select("div.colonne")
.map { article ->
val href = article.selectFirstOrThrow("a.index-top4-a").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
title = article.select(".carteinfos a").text(),
altTitle = null,
url = href,
publicUrl = href.toAbsoluteUrl(domain),
rating = RATING_UNKNOWN,
isNsfw = false,
coverUrl = article.selectFirstOrThrow("img.attachment-thumbnail").attrAsAbsoluteUrl("src"),
tags = setOf(),
state = null,
author = null,
source = source,
)
}
} else {
val root = doc.requireElementById("main")
return root.select("article.post-outer")
.map { article ->
val href = article.selectFirstOrThrow("a.thumb-link").attrAsRelativeUrl("href")
Manga(
id = generateUid(href),
title = article.select(".index-post-header a").text(),
altTitle = null,
url = href,
publicUrl = href.toAbsoluteUrl(domain),
rating = RATING_UNKNOWN,
isNsfw = false,
coverUrl = article.selectFirstOrThrow("img").attrAsAbsoluteUrl("src"),
tags = setOf(),
state = null,
author = null,
source = source,
)
}
}
}
override suspend fun getDetails(manga: Manga): Manga {
val root = webClient.httpGet(manga.url.toAbsoluteUrl(domain)).parseHtml().requireElementById("main")
val dateFormat = SimpleDateFormat("dd-MM-yyyy", Locale.FRANCE)
return manga.copy(
altTitle = root.select(".divider2:contains(Noms associés :)").firstOrNull()?.text(),
state = when (root.select(".label.label-primary")[2].text()) {
"En cours" -> MangaState.ONGOING
"Terminé", "Abondonné", "One Shot" -> MangaState.FINISHED
else -> null
},
tags = root.select("div.project-details a[href*=tag]").mapNotNullToSet { a ->
MangaTag(
key = a.attr("href").removeSuffix("/").substringAfterLast('/'),
title = a.text().toTitleCase(),
source = source,
)
},
author = root.select("div.project-details a[href*=auteur]").text(),
description = root.selectFirst("p.sContent")?.html(),
chapters = root.select("div.chapter-list li")
.mapChapters(reversed = true) { i, li ->
val href = li.getElementsByTag("a").firstNotNullOf { a ->
a.attrAsAbsoluteUrlOrNull("href")?.takeIf { it.contains("https://$domain/read/") }
}
val name = if (li.select(".chapter-name").isNullOrEmpty()) {
li.select(".chapter-name").text()
} else {
"Chapter $i"
}
val date = li.select(".name-chapter").first()!!.children().elementAt(2).text()
MangaChapter(
id = generateUid(href),
name = name,
number = i,
url = href,
scanlator = null,
uploadDate = dateFormat.tryParse(date),
branch = null,
source = source,
)
},
)
}
override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
val fullUrl = chapter.url.toAbsoluteUrl(domain)
val doc = webClient.httpGet(fullUrl).parseHtml()
val root = doc.body().selectFirst("#webtoon")
?: throw ParseException("Root not found", fullUrl)
return root.select("img").map { img ->
val url = img.attrAsRelativeUrlOrNull("data-src") ?: img.attrAsRelativeUrlOrNull("src")
?: img.parseFailed("Image src not found")
MangaPage(
id = generateUid(url),
url = url,
preview = null,
source = source,
)
}
}
override suspend fun getAvailableTags(): Set<MangaTag> {
val doc = webClient.httpGet("https://$domain/").parseHtml()
val body = doc.body()
val root = body.select(".asp_gochosen")[1]
val list = root?.select("option").orEmpty()
return list.mapToSet { li ->
MangaTag(
key = li.text(),
title = li.text(),
source = source,
)
}
}
}

Some files were not shown because too many files have changed in this diff Show More