[MangaTrend] New source (close #65)

[NetTruyen] New source

.gitignore

@@ -75,4 +75,5 @@ build/
 .idea/**/ktlint.xml
 .idea/codeStyles/
 
-src/test/resources/cookies.txt
+src/test/resources/cookies.txt
+local.properties

CONTRIBUTING.md

@@ -0,0 +1,90 @@
+# Contributing
+
+The following is guide for creating a Kotatsu parsers. Thanks for taking the time to contribute!
+
+## Prerequisites
+
+Before you start, please note that the ability to use following technologies is **required**.
+
+- Basic [Android development](https://developer.android.com/)
+- [Kotlin](https://kotlinlang.org/)
+- Web scraping ([JSoup](https://jsoup.org/)) or JSON API
+
+### Tools
+
+- [Android Studio](https://developer.android.com/studio)
+- [IntelliJ IDEA](https://www.jetbrains.com/idea/) (Community edition is enough)
+- Android device (or emulator)
+
+Kotatsu parsers is not a part of Android application, but you can easily develop and test it directly inside an Android
+application project and relocate it to the library project when done.
+
+### Before you start
+
+First, take a look at `kotatsu-parsers` project structure. Each parser is a single class that
+extends `MangaParser` class and have a `MangaSourceParser` annotation.
+Also pay attention on extensions in `util` package. For example, extensions from `Jsoup` file
+should be used instead of existing JSoup functions because they have better nullability support
+and improved error messages.
+
+## Writing your parser
+
+So, you want to create a parser, that will provide access to manga from a website.
+First, you should explore a website for API availability.
+If it does not contain any documentation about
+API, [explore network requests](https://firefox-source-docs.mozilla.org/devtools-user/):
+some websites use ajax.
+
+- [Example](https://github.com/KotatsuApp/kotatsu-parsers/blob/master/src/main/kotlin/org/koitharu/kotatsu/parsers/site/DesuMeParser.kt)
+  of Json API usage.
+- [Example](https://github.com/KotatsuApp/kotatsu-parsers/blob/master/src/main/kotlin/org/koitharu/kotatsu/parsers/site/AnibelParser.kt)
+  of GraphQL API usage
+- [Example](https://github.com/KotatsuApp/kotatsu-parsers/blob/master/src/main/kotlin/org/koitharu/kotatsu/parsers/site/MangaTownParser.kt)
+  of pure HTML parsing.
+
+If website is based on some engine it is rationally to use common base class for this one (for example, Madara wordress
+theme
+and the `MadaraParser` class)
+
+### Parser class skeleton
+
+Parser class must have exactly one primary constructor parameter of type `MangaLoaderContext` and have an
+`MangaSourceParser` annotation that provides internal name, title and language of a manga source.
+
+All functions in `MangaParser` class are documented. Pay attention to some peculiarities:
+
+- Never hardcode domain. Specify default domain in `configKeyDomain` field and obtain an actual one using `getDomain()`.
+- All ids must be unique and domain-independent. Use `generateUid` functions with relative url or some internal id which
+  is unique across the manga source.
+- `sortOrders` set should not be empty. If your source is not support sorting, specify one most relevance value.
+- If you cannot obtain direct links to pages images inside `getPages` method, it is ok to use an intermediate url
+  as `Page.url` and fetch a direct link at `getPageUrl` function.
+- `getFaviconUrl` function is deprecated, but Kotatsu application before v4 still use it, so it is recommended to
+  override it too.
+- You can use _asserts_ to check some optional fields. For example. `Manga.author` field is not required, but if your
+  source provide such information, add `assert(it != null)`. This will not have any effect on production but help to
+  find issues during unit testing.
+- If your source website (or it's api) uses pages for pagination instead of offset you should extend `PagedMangaParser`
+  instead of `MangaParser`.
+
+## Development process
+
+During the development it is recommended (but not necessary) to write it directly
+in the Kotatsu android application project. You can use `core.parser.DummyParser` class as a sandbox. `Dummy` manga
+source is available in debug Kotatsu build.
+
+Once parser is ready you can relocate your code into `kotatsu-parsers` library project in a `site` package and create a
+Pull Request.
+
+### Testing
+
+It is recommended to run unit tests before submitting a PR.
+
+- Temporary modify the `MangaSources` annotation class: specify your parser(s) name(s) and change mode
+  to `EnumSource.Mode.INCLUDE`
+- Run the `MangaParserTest` (`gradlew :test --tests "org.koitharu.kotatsu.parsers.MangaParserTest"`)
+- Optionally, you can run the `generateTestsReport` gradle task to get a pretty readable html report from test results.
+
+## Help
+
+If you need a help or have some questions, ask a community in our [Discord server](https://discord.gg/NNJ5RgVBC5).

README.md

@@ -4,7 +4,7 @@ Library that provides manga sources.
 
 [![](https://jitpack.io/v/KotatsuApp/kotatsu-parsers.svg)](https://jitpack.io/#KotatsuApp/kotatsu-parsers) ![Kotlin](https://img.shields.io/github/languages/top/KotatsuApp/kotatsu-parsers) ![License](https://img.shields.io/github/license/KotatsuApp/Kotatsu) [![Discord](https://img.shields.io/discord/898363402467045416?color=5865f2&label=discord)](https://discord.gg/NNJ5RgVBC5)
 
-### Usage
+## Usage
 
 1. Add it in your root build.gradle at the end of repositories:
 
@@ -48,4 +48,13 @@ Library that provides manga sources.
    and [Non-Android](https://github.com/KotatsuApp/kotatsu-dl/blob/master/src/main/kotlin/org/koitharu/kotatsu_dl/env/MangaLoaderContextImpl.kt)
    implementation examples.
 
-   Note that the `MangaSource.LOCAL` and `MangaSource.DUMMY` parsers cannot be instantiated.
+   Note that the `MangaSource.LOCAL` and `MangaSource.DUMMY` parsers cannot be instantiated.
+
+## Contribution
+
+See [CONTRIBUTING.md](./CONTRIBUTING.md) for the guidelines.
+
+## DMCA disclaimer
+
+The developers of this application does not have any affiliation with the content available in the app. It is collecting
+from the sources freely available through any web browser.

build.gradle

@@ -20,6 +20,7 @@ compileKotlin {
 		freeCompilerArgs += [
 				'-opt-in=kotlin.RequiresOptIn',
 				'-opt-in=kotlin.contracts.ExperimentalContracts',
+				'-opt-in=kotlinx.coroutines.ExperimentalCoroutinesApi',
 				'-opt-in=org.koitharu.kotatsu.parsers.InternalParsersApi',
 		]
 	}
@@ -62,9 +63,9 @@ dependencies {
 
 	ksp project(':kotatsu-parsers-ksp')
 
-	testImplementation 'org.junit.jupiter:junit-jupiter-api:5.8.2'
-	testImplementation 'org.junit.jupiter:junit-jupiter-engine:5.8.2'
-	testImplementation 'org.junit.jupiter:junit-jupiter-params:5.8.2'
+	testImplementation 'org.junit.jupiter:junit-jupiter-api:5.9.0'
+	testImplementation 'org.junit.jupiter:junit-jupiter-engine:5.9.0'
+	testImplementation 'org.junit.jupiter:junit-jupiter-params:5.9.0'
 	testImplementation 'org.jetbrains.kotlinx:kotlinx-coroutines-test:1.6.4'
 	testImplementation 'io.webfolder:quickjs:1.1.0'
 }

kotatsu-parsers-ksp/src/main/kotlin/org/koitharu/kotatsu/parsers/ksp/ParserProcessor.kt

@@ -69,6 +69,7 @@ class ParserProcessor(
 
 				import org.koitharu.kotatsu.parsers.model.MangaSource
 				
+				@Suppress("DEPRECATION")
 				fun MangaSource.newParser(context: MangaLoaderContext): MangaParser = when (this) {
 				
 			""".trimIndent(),
@@ -121,6 +122,7 @@ class ParserProcessor(
 				logger.error("Only non-abstract can be annotated with @MangaSourceParser", classDeclaration)
 			}
 			val annotation = classDeclaration.annotations.single { it.shortName.asString() == "MangaSourceParser" }
+			val deprecation = classDeclaration.annotations.singleOrNull { it.shortName.asString() == "Deprecated" }
 			val name = annotation.arguments.single { it.name?.asString() == "name" }.value as String
 			val title = annotation.arguments.single { it.name?.asString() == "title" }.value as String
 			val locale = annotation.arguments.single { it.name?.asString() == "locale" }.value as String
@@ -142,11 +144,15 @@ class ParserProcessor(
 					classDeclaration,
 				)
 			}
-
 			val className = classDeclaration.qualifiedName?.asString()
 			factoryWriter?.write("\tMangaSource.$name -> $className(context)\n")
+			val deprecationString = if (deprecation != null) {
+				val reason = deprecation.arguments
+					.find { it.name?.asString() == "message" }?.value?.toString() ?: "Unknown reason"
+				"@Deprecated(\"$reason\") "
+			} else ""
 			val localeComment = localeTitle?.toTitleCase(localeObj)?.let { " /* $it */" }.orEmpty()
-			sourcesWriter?.write("\t$name(\"$title\", $localeString$localeComment),\n")
+			sourcesWriter?.write("\t$deprecationString$name(\"$title\", $localeString$localeComment),\n")
 		}
 	}
 }

src/main/kotlin/org/koitharu/kotatsu/parsers/site/BlogTruyenParser.kt

@@ -0,0 +1,248 @@
+package org.koitharu.kotatsu.parsers.site
+
+import androidx.collection.ArrayMap
+import kotlinx.coroutines.sync.Mutex
+import kotlinx.coroutines.sync.withLock
+import org.json.JSONArray
+import org.jsoup.nodes.Document
+import org.jsoup.nodes.Element
+import org.koitharu.kotatsu.parsers.MangaLoaderContext
+import org.koitharu.kotatsu.parsers.MangaSourceParser
+import org.koitharu.kotatsu.parsers.PagedMangaParser
+import org.koitharu.kotatsu.parsers.config.ConfigKey
+import org.koitharu.kotatsu.parsers.model.*
+import org.koitharu.kotatsu.parsers.util.*
+import java.text.SimpleDateFormat
+import java.util.*
+import kotlin.collections.HashSet
+
+@MangaSourceParser("BLOGTRUYEN", "BlogTruyen", "vi")
+class BlogTruyenParser(override val context: MangaLoaderContext) :
+	PagedMangaParser(MangaSource.BLOGTRUYEN, pageSize = 20) {
+
+	override val configKeyDomain: ConfigKey.Domain
+		get() = ConfigKey.Domain("blogtruyen.vn", null)
+
+	override val sortOrders: Set<SortOrder>
+		get() = EnumSet.of(SortOrder.UPDATED)
+
+	private val mutex = Mutex()
+	private val dateFormat = SimpleDateFormat("dd/MM/yyyy HH:mm", Locale.US)
+	private var cacheTags: ArrayMap<String, MangaTag>? = null
+
+	override suspend fun getDetails(manga: Manga): Manga {
+		val doc = context.httpGet(manga.url.toAbsoluteUrl(getDomain())).parseHtml()
+		val descriptionElement = doc.selectFirstOrThrow("div.description")
+		val statusText = descriptionElement
+			.selectFirst("p:contains(Trạng thái) > span.color-red")
+			?.text()
+
+		val state = when (statusText) {
+			"Đang tiến hành" -> MangaState.ONGOING
+			"Đã hoàn thành" -> MangaState.FINISHED
+			else -> null
+		}
+
+		val rating = doc.selectFirst("span.total-vote")?.attr("ng-init")?.let { text ->
+			val like = text.substringAfter("TotalLike=")
+				.substringBefore(';')
+				.toIntOrNull() ?: return@let RATING_UNKNOWN
+			val dislike = text.substringAfter("TotalDisLike=")
+				.toIntOrNull() ?: return@let RATING_UNKNOWN
+
+			when {
+				like == 0 && dislike == 0 -> RATING_UNKNOWN
+				else -> like.toFloat() / (like + dislike)
+			}
+		}
+
+		val tagMap = getOrCreateTagMap()
+		val tags = descriptionElement.select("p > span.category").mapNotNullToSet {
+			val tagName = it.selectFirst("a")?.text()?.trim() ?: return@mapNotNullToSet null
+			tagMap[tagName]
+		}
+
+		return manga.copy(
+			tags = tags,
+			author = descriptionElement.selectFirst("p:contains(Tác giả) > a")?.text(),
+			description = doc.selectFirst(".detail .content")?.html(),
+			chapters = parseChapterList(doc),
+			largeCoverUrl = doc.selectLast("div.thumbnail > img")?.attrAsAbsoluteUrlOrNull("src"),
+			state = state,
+			rating = rating ?: RATING_UNKNOWN,
+			isNsfw = doc.getElementById("warningCategory") != null
+		)
+	}
+
+	private fun parseChapterList(doc: Document): List<MangaChapter> {
+		val chapterList = doc.select("#list-chapters > p")
+		return chapterList.asReversed().mapChapters { index, element ->
+			val titleElement = element.selectFirst("span.title > a") ?: return@mapChapters null
+			val name = titleElement.text()
+			val relativeUrl = titleElement.attrAsRelativeUrl("href")
+			val id = relativeUrl.substringAfter('/').substringBefore('/')
+			val uploadDate = dateFormat.tryParse(element.select("span.publishedDate").text())
+			MangaChapter(
+				id = generateUid(id),
+				name = name,
+				number = index + 1,
+				url = relativeUrl,
+				scanlator = null,
+				uploadDate = uploadDate,
+				branch = null,
+				source = source
+			)
+		}
+	}
+
+	override suspend fun getListPage(
+		page: Int,
+		query: String?,
+		tags: Set<MangaTag>?,
+		sortOrder: SortOrder,
+	): List<Manga> {
+		return when {
+			!query.isNullOrEmpty() -> {
+				val searchUrl = "https://${getDomain()}/timkiem/nangcao/1/0/-1/-1?txt=${query.urlEncoded()}&p=$page"
+				val searchContent = context.httpGet(searchUrl).parseHtml()
+					.selectFirst("section.list-manga-bycate > div.list")
+				parseMangaList(searchContent)
+			}
+
+			!tags.isNullOrEmpty() -> {
+				val tag = tags.oneOrThrowIfMany()!!
+				val categoryAjax = "https://${getDomain()}/ajax/Category/AjaxLoadMangaByCategory?id=${tag.key}&orderBy=5&p=$page"
+				val listContent = context.httpGet(categoryAjax).parseHtml().selectFirst("div.list")
+				parseMangaList(listContent)
+			}
+			else -> getNormalList(page)
+		}
+	}
+
+	private suspend fun getNormalList(page: Int): List<Manga> {
+		val pageLink = "https://${getDomain()}/page-$page"
+		val doc = context.httpGet(pageLink).parseHtml()
+		val listElements = doc.selectFirstOrThrow("section.list-mainpage.listview")
+			.select("div.bg-white.storyitem")
+
+		return listElements.mapNotNull {
+			val linkTag = it.selectFirst("div.fl-l > a") ?: return@mapNotNull null
+			val relativeUrl = linkTag.attrAsRelativeUrl("href")
+			val tagMap = getOrCreateTagMap()
+			val tags = it.select("footer > div.category > a").mapNotNullToSet { a ->
+				tagMap[a.text()]
+			}
+
+			Manga(
+				id = generateUid(relativeUrl),
+				title = linkTag.attr("title"),
+				altTitle = null,
+				description = it.selectFirst("p.al-j.break.line-height-15")?.text(),
+				url = relativeUrl,
+				publicUrl = relativeUrl.toAbsoluteUrl(getDomain()),
+				coverUrl = linkTag.selectLast("img")?.attr("src").orEmpty(),
+				source = source,
+				tags = tags,
+				isNsfw = false,
+				rating = RATING_UNKNOWN,
+				author = null,
+				state = null,
+			)
+		}
+	}
+
+	private fun parseMangaList(listElement: Element?): List<Manga> {
+		listElement ?: return emptyList()
+
+		return listElement.select("span.tiptip[data-tiptip]").mapNotNull {
+			val mangaInfo = listElement.getElementById(it.attr("data-tiptip")) ?: return@mapNotNull null
+			val a = it.selectFirst("a") ?: return@mapNotNull null
+			val relativeUrl = a.attrAsRelativeUrl("href")
+			Manga(
+				id = generateUid(relativeUrl),
+				title = a.text(),
+				altTitle = null,
+				description = mangaInfo.select("div.al-j.fs-12").text(),
+				url = relativeUrl,
+				publicUrl = relativeUrl.toAbsoluteUrl(getDomain()),
+				coverUrl = mangaInfo.selectFirst("div > img.img")?.absUrl("src").orEmpty(),
+				isNsfw = false,
+				rating = RATING_UNKNOWN,
+				tags = emptySet(),
+				author = null,
+				state = null,
+				source = source,
+			)
+		}
+	}
+
+	override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
+		fun generateImageId(index: Int) = generateUid("${chapter.url}/$index")
+
+		val doc = context.httpGet(chapter.url.toAbsoluteUrl(getDomain())).parseHtml()
+		val pages = ArrayList<MangaPage>()
+		val referer = chapter.url.toAbsoluteUrl(getDomain())
+		doc.select("#content > img").forEach { img ->
+			val url = img.attrAsRelativeUrl("src")
+			pages.add(
+				MangaPage(
+					id = generateImageId(pages.lastIndex),
+					url = url,
+					referer = referer,
+					preview = null,
+					source = source,
+				)
+			)
+		}
+
+		// Some chapters use js script to render images
+		val script = doc.selectLast("#content > script")
+		if (script != null && script.data().contains("listImageCaption")) {
+			val imagesStr = script.data().substringBefore(';').substringAfterLast('=').trim()
+			val imageArr = JSONArray(imagesStr)
+			for (i in 0 until imageArr.length()) {
+				val imageUrl = imageArr.getJSONObject(i).getString("url")
+				pages.add(
+					MangaPage(
+						id = generateImageId(pages.lastIndex),
+						url = imageUrl,
+						referer = referer,
+						preview = null,
+						source = source
+					)
+				)
+			}
+		}
+
+		return pages
+	}
+
+	override suspend fun getTags(): Set<MangaTag> {
+		val map = getOrCreateTagMap()
+		val tags = HashSet<MangaTag>(map.size)
+		for (entry in map) {
+			tags.add(entry.value)
+		}
+
+		return tags
+	}
+
+
+	private suspend fun getOrCreateTagMap(): ArrayMap<String, MangaTag> = mutex.withLock {
+		cacheTags?.let { return@withLock it }
+		val doc = context.httpGet("/timkiem/nangcao".toAbsoluteUrl(getDomain())).parseHtml()
+		val tagItems = doc.select("li[data-id]")
+		val tagMap = ArrayMap<String, MangaTag>(tagItems.size)
+		for (tag in tagItems) {
+			val title = tag.text().trim()
+			tagMap[tag.text().trim()] = MangaTag(
+				title = title,
+				key = tag.attr("data-id"),
+				source = source
+			)
+		}
+
+		cacheTags = tagMap
+		tagMap
+	}
+}

src/main/kotlin/org/koitharu/kotatsu/parsers/site/CloneMangaParser.kt

@@ -0,0 +1,104 @@
+package org.koitharu.kotatsu.parsers.site
+
+import org.koitharu.kotatsu.parsers.*
+import org.koitharu.kotatsu.parsers.config.ConfigKey
+import org.koitharu.kotatsu.parsers.model.*
+import org.koitharu.kotatsu.parsers.util.*
+import java.util.*
+
+@MangaSourceParser("CLONEMANGA", "CloneManga", "en")
+internal class CloneMangaParser(override val context: MangaLoaderContext) : PagedMangaParser(
+	MangaSource.CLONEMANGA,
+	pageSize = 1,
+) {
+
+	override val sortOrders: Set<SortOrder> = Collections.singleton(
+		SortOrder.POPULARITY,
+	)
+
+	override val configKeyDomain = ConfigKey.Domain("manga.clone-army.org", null)
+
+	override fun getFaviconUrl(): String {
+		return "https://pbs.twimg.com/profile_images/458758466346029056/Ys93EANp_400x400.png"
+	}
+
+	override suspend fun getListPage(
+		page: Int,
+		query: String?,
+		tags: Set<MangaTag>?,
+		sortOrder: SortOrder,
+	): List<Manga> {
+		if (query != null || page > 1) {
+			return emptyList()
+		}
+		val link = "https://${getDomain()}/viewer_landing.php"
+		val doc = context.httpGet(link).parseHtml()
+		val mangas = doc.getElementsByClass("comicPreviewContainer")
+		return mangas.mapNotNull { item ->
+			val attr = item.getElementsByClass("comicPreview").attr("style")
+			val href = item.selectFirst("a")?.attrAsAbsoluteUrl("href") ?: return@mapNotNull null
+			val cover = attr.substring(attr.indexOf("site/themes"), attr.indexOf(")"))
+			Manga(
+				id = generateUid(href),
+				title = item.selectFirst("h3")?.text() ?: return@mapNotNull null,
+				coverUrl = "https://${getDomain()}/$cover",
+				altTitle = null,
+				author = "Dan Kim",
+				rating = RATING_UNKNOWN,
+				url = href,
+				isNsfw = false,
+				tags = emptySet(),
+				state = null,
+				publicUrl = href.toAbsoluteUrl(getDomain()),
+				source = source,
+			)
+		}
+	}
+
+	override suspend fun getDetails(manga: Manga): Manga {
+		val doc = context.httpGet(manga.publicUrl).parseHtml()
+		val series = doc.location()
+		val numChapters = Regex(
+			pattern = "&page=(.*)&lang=",
+		).findAll(
+			input = doc.getElementsByTag("script")[3].toString(),
+		)
+			.elementAt(3).destructured.component1()
+			.toInt()
+		val chapters = ArrayList<MangaChapter>()
+		for (i in 0..numChapters) {
+			val chapter = MangaChapter(
+				id = generateUid("$series&page=$i"),
+				name = "Chapter ${i + 1}",
+				number = i + 1,
+				url = "$series&page=$i",
+				scanlator = null,
+				branch = null,
+				uploadDate = 0L,
+				source = MangaSource.DUMMY,
+			)
+			chapters.add(chapter)
+		}
+		return manga.copy(chapters = chapters)
+	}
+
+	override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
+		val doc = context.httpGet(chapter.url.toAbsoluteUrl(getDomain())).parseHtml()
+		val imgUrl = doc.getElementsByClass("subsectionContainer")[0]
+			.selectFirst("img")
+			?.attrAsAbsoluteUrlOrNull("src") ?: doc.parseFailed("Something broken")
+		return listOf(
+			MangaPage(
+				id = generateUid(imgUrl),
+				url = imgUrl,
+				referer = imgUrl,
+				preview = null,
+				source = MangaSource.DUMMY,
+			),
+		)
+	}
+
+	override suspend fun getTags(): Set<MangaTag> {
+		return emptySet()
+	}
+}

src/main/kotlin/org/koitharu/kotatsu/parsers/site/MangaDexParser.kt

@@ -1,12 +1,15 @@
 package org.koitharu.kotatsu.parsers.site
 
+import kotlinx.coroutines.Dispatchers
 import kotlinx.coroutines.async
+import kotlinx.coroutines.awaitAll
 import kotlinx.coroutines.coroutineScope
 import org.json.JSONObject
 import org.koitharu.kotatsu.parsers.MangaLoaderContext
 import org.koitharu.kotatsu.parsers.MangaParser
 import org.koitharu.kotatsu.parsers.MangaSourceParser
 import org.koitharu.kotatsu.parsers.config.ConfigKey
+import org.koitharu.kotatsu.parsers.exception.ParseException
 import org.koitharu.kotatsu.parsers.model.*
 import org.koitharu.kotatsu.parsers.util.*
 import org.koitharu.kotatsu.parsers.util.json.*
@@ -14,7 +17,9 @@ import java.text.SimpleDateFormat
 import java.util.*
 
 private const val PAGE_SIZE = 20
-private const val CHAPTERS_PAGE_SIZE = 60
+private const val CHAPTERS_FIRST_PAGE_SIZE = 120
+private const val CHAPTERS_MAX_PAGE_SIZE = 500
+private const val CHAPTERS_PARALLELISM = 3
 private const val CONTENT_RATING =
 	"contentRating[]=safe&contentRating[]=suggestive&contentRating[]=erotica&contentRating[]=pornographic"
 private const val LOCALE_FALLBACK = "en"
@@ -62,6 +67,7 @@ internal class MangaDexParser(override val context: MangaLoaderContext) : MangaP
 				when (sortOrder) {
 					SortOrder.UPDATED,
 					-> "[latestUploadedChapter]=desc"
+
 					SortOrder.ALPHABETICAL -> "[title]=asc"
 					SortOrder.NEWEST -> "[createdAt]=desc"
 					SortOrder.POPULARITY -> "[followedCount]=desc"
@@ -118,12 +124,13 @@ internal class MangaDexParser(override val context: MangaLoaderContext) : MangaP
 
 	override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
 		val domain = getDomain()
+		val mangaId = manga.url.removePrefix("/")
 		val attrsDeferred = async {
 			context.httpGet(
-				"https://api.$domain/manga/${manga.url}?includes[]=artist&includes[]=author&includes[]=cover_art",
+				"https://api.$domain/manga/${mangaId}?includes[]=artist&includes[]=author&includes[]=cover_art",
 			).parseJson().getJSONObject("data").getJSONObject("attributes")
 		}
-		val feedDeferred = async { loadChapters(manga) }
+		val feedDeferred = async { loadChapters(mangaId) }
 		val mangaAttrs = attrsDeferred.await()
 		val feed = feedDeferred.await()
 		// 2022-01-02T00:27:11+00:00
@@ -134,11 +141,11 @@ internal class MangaDexParser(override val context: MangaLoaderContext) : MangaP
 		manga.copy(
 			description = mangaAttrs.getJSONObject("description").selectByLocale()
 				?: manga.description,
-			chapters = feed.mapNotNull { jo ->
+			chapters = feed.mapChapters { _, jo ->
 				val id = jo.getString("id")
 				val attrs = jo.getJSONObject("attributes")
 				if (!attrs.isNull("externalUrl")) {
-					return@mapNotNull null
+					return@mapChapters null
 				}
 				val locale = attrs.getStringOrNull("translatedLanguage")?.let { Locale.forLanguageTag(it) }
 				val relations = jo.getJSONArray("relationships").associateByKey("type")
@@ -201,30 +208,62 @@ internal class MangaDexParser(override val context: MangaLoaderContext) : MangaP
 		return getStringOrNull(LOCALE_FALLBACK) ?: values().nextOrNull() as? String
 	}
 
-	private suspend fun loadChapters(manga: Manga): List<JSONObject> {
-		val domain = getDomain()
-		val result = ArrayList<JSONObject>()
-		while (true) {
-			val url = buildString {
-				append("https://api.")
-				append(domain)
-				append("/manga/")
-				append(manga.url)
-				append("/feed")
-				append("?limit=")
-				append(CHAPTERS_PAGE_SIZE)
-				append("&includes[]=scanlation_group&order[volume]=asc&order[chapter]=asc&offset=")
-				append(result.size)
-				append('&')
-				append(CONTENT_RATING)
-			}
-			val json = context.httpGet(url).parseJson()
-			val data = json.optJSONArray("data")
-			if (data.isNullOrEmpty()) {
-				break
-			}
-			data.mapJSONTo(result) { it }
+	private suspend fun loadChapters(mangaId: String): List<JSONObject> {
+		val firstPage = loadChapters(mangaId, offset = 0, limit = CHAPTERS_FIRST_PAGE_SIZE)
+		if (firstPage.size >= firstPage.total) {
+			return firstPage.data
 		}
+		val tail = coroutineScope {
+			val leftCount = firstPage.total - firstPage.size
+			val pages = (leftCount / CHAPTERS_MAX_PAGE_SIZE.toFloat()).toIntUp()
+			val dispatcher = Dispatchers.Default.limitedParallelism(CHAPTERS_PARALLELISM)
+			List(pages) { page ->
+				val offset = page * CHAPTERS_MAX_PAGE_SIZE + firstPage.size
+				async(dispatcher) {
+					loadChapters(mangaId, offset, CHAPTERS_MAX_PAGE_SIZE)
+				}
+			}.awaitAll()
+		}
+		val result = ArrayList<JSONObject>(firstPage.total)
+		result += firstPage.data
+		tail.flatMapTo(result) { it.data }
 		return result
 	}
+
+	private suspend fun loadChapters(mangaId: String, offset: Int, limit: Int): Chapters {
+		val url = buildString {
+			append("https://api.")
+			append(getDomain())
+			append("/manga/")
+			append(mangaId)
+			append("/feed")
+			append("?limit=")
+			append(limit)
+			append("&includes[]=scanlation_group&order[volume]=asc&order[chapter]=asc&offset=")
+			append(offset)
+			append('&')
+			append(CONTENT_RATING)
+		}
+		val json = context.httpGet(url).parseJson()
+		if (json.getString("result") == "ok") {
+			return Chapters(
+				data = json.optJSONArray("data")?.toJSONList().orEmpty(),
+				total = json.getInt("total"),
+			)
+		} else {
+			val error = json.optJSONArray("errors").mapJSON { jo ->
+				jo.getString("detail")
+			}.joinToString("\n")
+			throw ParseException(error, url)
+		}
+	}
+
+	private class Chapters(
+		val data: List<JSONObject>,
+		val total: Int,
+	) {
+
+		val size: Int
+			get() = data.size
+	}
 }

src/main/kotlin/org/koitharu/kotatsu/parsers/site/MangaInUaParser.kt

@@ -83,9 +83,7 @@ class MangaInUaParser(override val context: MangaLoaderContext) : PagedMangaPars
 		val doc = context.httpGet(manga.url.toAbsoluteUrl(getDomain())).parseHtml()
 		val root = doc.body().requireElementById("dle-content")
 		val dateFormat = SimpleDateFormat("dd.MM.yyyy", Locale.US)
-		val chapterNodes = root.selectFirstOrThrow(".linkstocomics")
-			.select(".ltcitems")
-			.filterNot { x -> x.selectFirst(".foruserreadedbar") != null }
+		val chapterNodes = root.selectFirstOrThrow(".linkstocomics").select(".ltcitems")
 		var prevChapterName: String? = null
 		var i = 0
 		return manga.copy(

src/main/kotlin/org/koitharu/kotatsu/parsers/site/MangaOwlParser.kt

@@ -9,6 +9,7 @@ import org.koitharu.kotatsu.parsers.util.*
 import java.text.SimpleDateFormat
 import java.util.*
 
+@Deprecated("Permanently shutdown")
 @MangaSourceParser("MANGAOWL", "MangaOwl", "en")
 internal class MangaOwlParser(override val context: MangaLoaderContext) : MangaParser(MangaSource.MANGAOWL) {
 

src/main/kotlin/org/koitharu/kotatsu/parsers/site/NetTruyenParser.kt

@@ -0,0 +1,221 @@
+package org.koitharu.kotatsu.parsers.site
+
+import androidx.collection.ArrayMap
+import androidx.collection.ArraySet
+import kotlinx.coroutines.sync.Mutex
+import kotlinx.coroutines.sync.withLock
+import org.koitharu.kotatsu.parsers.*
+import org.koitharu.kotatsu.parsers.config.ConfigKey
+import org.koitharu.kotatsu.parsers.exception.NotFoundException
+import org.koitharu.kotatsu.parsers.model.*
+import org.koitharu.kotatsu.parsers.util.*
+import java.text.SimpleDateFormat
+import java.util.*
+
+@MangaSourceParser("NETTRUYEN", "NetTruyen", "vi")
+class NetTruyenParser(override val context: MangaLoaderContext) :
+	PagedMangaParser(MangaSource.NETTRUYEN, pageSize = 36) {
+
+	override val configKeyDomain: ConfigKey.Domain
+		get() = ConfigKey.Domain("www.nettruyenme.com", null)
+
+	override val sortOrders: Set<SortOrder>
+		get() = EnumSet.of(SortOrder.UPDATED, SortOrder.POPULARITY, SortOrder.NEWEST, SortOrder.RATING)
+
+	private val mutex = Mutex()
+	private val dateFormat = SimpleDateFormat("dd/MM/yy", Locale.US)
+	private var tagCache: ArrayMap<String, MangaTag>? = null
+
+	override suspend fun getDetails(manga: Manga): Manga {
+		val doc = context.httpGet(manga.url.toAbsoluteUrl(getDomain())).parseHtml()
+		val rating = doc.selectFirst("span[itemprop=ratingValue]")
+			?.ownText()
+			?.toFloatOrNull() ?: 0f
+
+		val chapterElements = doc.getElementById("nt_listchapter")?.select("ul > li") ?: doc.parseFailed()
+		val chapters = chapterElements.asReversed().mapChapters { index, element ->
+			val a = element.selectFirst("div.chapter > a") ?: return@mapChapters null
+			val relativeUrl = a.attrAsRelativeUrlOrNull("href") ?: return@mapChapters null
+			val timeText = element.selectFirst("div.col-xs-4.text-center.no-wrap.small")?.text()
+
+			MangaChapter(
+				id = generateUid(relativeUrl),
+				name = a.text(),
+				number = index + 1,
+				url = relativeUrl,
+				scanlator = null,
+				uploadDate = parseChapterTime(timeText),
+				branch = null,
+				source = source,
+			)
+		}
+
+		return manga.copy(
+			rating = rating / 5,
+			chapters = chapters,
+			description = doc.selectFirst("div.detail-content > p")?.html(),
+			isNsfw = doc.selectFirst("div.alert.alert-danger > strong:contains(Cảnh báo độ tuổi)") != null,
+		)
+	}
+
+	// 20 giây trước
+	// 52 phút trước
+	// 6 giờ trước
+	// 2 ngày trước
+	// 19:09 30/07
+	// 23/12/21
+	private fun parseChapterTime(timeText: String?): Long {
+		if (timeText.isNullOrEmpty()) {
+			return 0L
+		}
+
+		val timeWords = arrayOf("giây", "phút", "giờ", "ngày")
+		val calendar = Calendar.getInstance()
+		val timeArr = timeText.split(' ')
+		if (WordSet(*timeWords).anyWordIn(timeText)) {
+			val timeSuffix = timeArr.getOrNull(1)
+			val timeDiff = timeArr.getOrNull(0)?.toIntOrNull() ?: return 0L
+			when (timeSuffix) {
+				timeWords[0] -> calendar.add(Calendar.SECOND, -timeDiff)
+				timeWords[1] -> calendar.add(Calendar.MINUTE, -timeDiff)
+				timeWords[2] -> calendar.add(Calendar.HOUR, -timeDiff)
+				timeWords[3] -> calendar.add(Calendar.DATE, -timeDiff)
+				else -> return 0L
+			}
+		} else {
+			val relativeDate = timeArr.lastOrNull() ?: return 0L
+			val dateString = when (relativeDate.split('/').size) {
+				2 -> {
+					val currentYear = calendar.get(Calendar.YEAR).toString().takeLast(2)
+					"$relativeDate/$currentYear"
+				}
+				3 -> relativeDate
+				else -> return 0L
+			}
+
+			calendar.timeInMillis = dateFormat.tryParse(dateString)
+		}
+
+
+		return calendar.time.time
+	}
+
+	override suspend fun getListPage(
+		page: Int,
+		query: String?,
+		tags: Set<MangaTag>?,
+		sortOrder: SortOrder,
+	): List<Manga> {
+		val isSearching = !query.isNullOrEmpty()
+		val url = buildString {
+			append("https://")
+			append(getDomain())
+			if (isSearching) {
+				append("/tim-truyen?keyword=")
+				append(query!!.urlEncoded())
+				append("&page=")
+				append(page)
+			} else {
+				val tagQuery = tags.orEmpty().joinToString(",") { it.key }
+				append("/tim-truyen-nang-cao?genres=$tagQuery")
+				append("&notgenres=&gender=-1&status=-1&minchapter=1&sort=${getSortOrderKey(sortOrder)}")
+				append("&page=$page")
+			}
+		}
+
+		val response = if (isSearching) {
+			val result = runCatching { context.httpGet(url) }
+			val exception = result.exceptionOrNull()
+			if (exception is NotFoundException) {
+				return emptyList()
+			}
+
+			result.getOrThrow()
+		} else {
+			context.httpGet(url)
+		}
+
+		val itemsElements = response.parseHtml()
+			.select("div.ModuleContent > div.items")
+			.select("div.item")
+		return itemsElements.mapNotNull { item ->
+			val tooltipElement = item.selectFirst("div.box_tootip") ?: return@mapNotNull null
+			val absUrl = item.selectFirst("div.image > a")?.attrAsAbsoluteUrlOrNull("href") ?: return@mapNotNull null
+			val slug = absUrl.substringAfterLast('/')
+			val mangaState = when (tooltipElement.selectFirst("div.message_main > p:contains(Tình trạng)")?.ownText()) {
+				"Đang tiến hành" -> MangaState.ONGOING
+				"Hoàn thành" -> MangaState.FINISHED
+				else -> null
+			}
+
+			val tagMap = getOrCreateTagMap()
+			val tagsElement = tooltipElement.selectFirst("div.message_main > p:contains(Thể loại)")?.ownText().orEmpty()
+			val mangaTags = tagsElement.split(',').mapNotNullToSet { tagMap[it.trim()] }
+			Manga(
+				id = generateUid(slug),
+				title = tooltipElement.selectFirst("div.title")?.text().orEmpty(),
+				altTitle = null,
+				url = absUrl.toRelativeUrl(getDomain()),
+				publicUrl = absUrl,
+				rating = RATING_UNKNOWN,
+				isNsfw = false,
+				coverUrl = item.selectFirst("div.image a img")?.absUrl("data-original").orEmpty(),
+				largeCoverUrl = null,
+				tags = mangaTags,
+				state = mangaState,
+				author = tooltipElement.selectFirst("div.message_main > p:contains(Tác giả)")?.ownText(),
+				description = tooltipElement.selectFirst("div.box_text")?.text(),
+				chapters = null,
+				source = source,
+			)
+		}
+	}
+
+
+	override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
+		val pageElements = context.httpGet(chapter.url.toAbsoluteUrl(getDomain())).parseHtml()
+			.select("div.reading-detail.box_doc > div img")
+		return pageElements.map { element ->
+			val url = element.attrAsAbsoluteUrl("data-original")
+			MangaPage(
+				id = generateUid(url),
+				url = url,
+				referer = getDomain(),
+				preview = null,
+				source = source,
+			)
+		}
+	}
+
+	override suspend fun getTags(): Set<MangaTag> {
+		val map = getOrCreateTagMap()
+		val tagSet = ArraySet<MangaTag>(map.size)
+		for (entry in map) {
+			tagSet.add(entry.value)
+		}
+
+		return tagSet
+	}
+
+	private suspend fun getOrCreateTagMap(): ArrayMap<String, MangaTag> = mutex.withLock {
+		tagCache?.let { return@withLock it }
+		val doc = context.httpGet("/tim-truyen-nang-cao".toAbsoluteUrl(getDomain())).parseHtml()
+		val tagItems = doc.select("div.genre-item")
+		val result = ArrayMap<String, MangaTag>(tagItems.size)
+		for (item in tagItems) {
+			val title = item.text().trim()
+			val key = item.select("span[data-id]").attr("data-id")
+			result[title] = MangaTag(title = title, key = key, source = source)
+		}
+		tagCache = result
+		result
+	}
+
+	private fun getSortOrderKey(sortOrder: SortOrder) = when (sortOrder) {
+		SortOrder.UPDATED -> 0
+		SortOrder.POPULARITY -> 10
+		SortOrder.NEWEST -> 15
+		SortOrder.RATING -> 20
+		else -> throw IllegalArgumentException("Sort order ${sortOrder.name} not supported")
+	}
+}

src/main/kotlin/org/koitharu/kotatsu/parsers/site/RemangaParser.kt

@@ -15,10 +15,7 @@ import org.koitharu.kotatsu.parsers.exception.ContentUnavailableException
 import org.koitharu.kotatsu.parsers.exception.ParseException
 import org.koitharu.kotatsu.parsers.model.*
 import org.koitharu.kotatsu.parsers.util.*
-import org.koitharu.kotatsu.parsers.util.json.getStringOrNull
-import org.koitharu.kotatsu.parsers.util.json.mapJSON
-import org.koitharu.kotatsu.parsers.util.json.mapJSONTo
-import org.koitharu.kotatsu.parsers.util.json.mapJSONToSet
+import org.koitharu.kotatsu.parsers.util.json.*
 import java.net.HttpURLConnection
 import java.net.URLDecoder
 import java.text.DateFormat
@@ -143,6 +140,9 @@ internal class RemangaParser(
 				)
 			},
 			chapters = chapters.mapChapters { i, jo ->
+				if (jo.getBooleanOrDefault("is_paid", false)) {
+					return@mapChapters null
+				}
 				val id = jo.getLong("id")
 				val name = jo.getString("name").toTitleCase(Locale.ROOT)
 				val publishers = jo.optJSONArray("publishers")

src/main/kotlin/org/koitharu/kotatsu/parsers/site/UnionMangasParser.kt

@@ -101,7 +101,7 @@ class UnionMangasParser(override val context: MangaLoaderContext) : PagedMangaPa
 					url = href,
 					scanlator = div.selectLast("a")?.text()?.takeUnless { it == title },
 					uploadDate = dateFormat.tryParse(
-						a.nextElementSibling()?.text()?.removeSurrounding('(', ')'),
+						a.nextElementSibling()?.text()?.removeSurrounding("(", ")"),
 					),
 					branch = null,
 					source = source,

src/main/kotlin/org/koitharu/kotatsu/parsers/site/grouple/GroupleParser.kt

@@ -111,11 +111,10 @@ internal abstract class GroupleParser(
 				altTitle = descDiv.selectFirst("h4")?.text(),
 				coverUrl = imgDiv.selectFirst("img.lazy")?.attr("data-original")?.replace("_p.", ".").orEmpty(),
 				rating = runCatching {
-					node.selectFirst("div.rating")
+					node.selectFirst(".compact-rate")
 						?.attr("title")
-						?.substringBefore(' ')
 						?.toFloatOrNull()
-						?.div(10f)
+						?.div(5f)
 				}.getOrNull() ?: RATING_UNKNOWN,
 				author = tileInfo?.selectFirst("a.person-link")?.text(),
 				isNsfw = false,

src/main/kotlin/org/koitharu/kotatsu/parsers/site/madara/IsekaiScanEu.kt

@@ -0,0 +1,47 @@
+package org.koitharu.kotatsu.parsers.site.madara
+
+import org.jsoup.nodes.Document
+import org.koitharu.kotatsu.parsers.MangaLoaderContext
+import org.koitharu.kotatsu.parsers.MangaSourceParser
+import org.koitharu.kotatsu.parsers.model.Manga
+import org.koitharu.kotatsu.parsers.model.MangaChapter
+import org.koitharu.kotatsu.parsers.model.MangaSource
+import org.koitharu.kotatsu.parsers.util.*
+import java.text.SimpleDateFormat
+import java.util.*
+
+@MangaSourceParser("ISEKAISCAN_EU", "IsekaiScan (eu)", "en")
+internal class IsekaiScanEuParser(context: MangaLoaderContext) :
+	MadaraParser(context, MangaSource.ISEKAISCAN_EU, "isekaiscan.eu") {
+
+	override val datePattern = "MM/dd/yyyy"
+
+	override suspend fun getChapters(manga: Manga, doc: Document): List<MangaChapter> {
+		val mangaId = doc.body().requireElementById("manga-chapters-holder").attr("data-id")
+		val ul = context.httpPost(
+			"https://${getDomain()}/wp-admin/admin-ajax.php",
+			mapOf(
+				"action" to "manga_get_chapters",
+				"manga" to mangaId,
+			),
+		).parseHtml().body().selectFirstOrThrow("ul")
+		val dateFormat = SimpleDateFormat(datePattern, Locale.US)
+		return ul.select("li").asReversed().mapChapters { i, li ->
+			val a = li.selectFirst("a")
+			val href = a?.attrAsRelativeUrlOrNull("href") ?: li.parseFailed("Link is missing")
+			MangaChapter(
+				id = generateUid(href),
+				name = a.ownText(),
+				number = i + 1,
+				url = href,
+				uploadDate = parseChapterDate(
+					dateFormat,
+					li.selectFirst("span.chapter-release-date i")?.text(),
+				),
+				source = source,
+				scanlator = null,
+				branch = null,
+			)
+		}
+	}
+}

src/main/kotlin/org/koitharu/kotatsu/parsers/site/madara/MadaraParser.kt

@@ -1,5 +1,8 @@
 package org.koitharu.kotatsu.parsers.site.madara
 
+import kotlinx.coroutines.async
+import kotlinx.coroutines.coroutineScope
+import org.jsoup.nodes.Document
 import org.jsoup.nodes.Element
 import org.koitharu.kotatsu.parsers.MangaLoaderContext
 import org.koitharu.kotatsu.parsers.MangaSourceParser
@@ -116,9 +119,10 @@ internal abstract class MadaraParser(
 		}
 	}
 
-	override suspend fun getDetails(manga: Manga): Manga {
+	override suspend fun getDetails(manga: Manga): Manga = coroutineScope {
 		val fullUrl = manga.url.toAbsoluteUrl(getDomain())
 		val doc = context.httpGet(fullUrl).parseHtml()
+		val chaptersDeferred = async { getChapters(manga, doc) }
 		val root = doc.body().selectFirst("div.profile-manga")
 			?.selectFirst("div.summary_content")
 			?.selectFirst("div.post-content")
@@ -126,8 +130,7 @@ internal abstract class MadaraParser(
 		val root2 = doc.body().selectFirst("div.content-area")
 			?.selectFirst("div.c-page")
 			?: throw ParseException("Root2 not found", fullUrl)
-		val dateFormat = SimpleDateFormat(datePattern, Locale.US)
-		return manga.copy(
+		manga.copy(
 			tags = root.selectFirst("div.genres-content")?.select("a")
 				?.mapNotNullToSet { a ->
 					MangaTag(
@@ -141,26 +144,33 @@ internal abstract class MadaraParser(
 				?.select("p")
 				?.filterNot { it.ownText().startsWith("A brief description") }
 				?.joinToString { it.html() },
-			chapters = root2.select("li").asReversed().mapChapters { i, li ->
-				val a = li.selectFirst("a")
-				val href = a?.attrAsRelativeUrlOrNull("href") ?: li.parseFailed("Link is missing")
-				MangaChapter(
-					id = generateUid(href),
-					name = a.ownText(),
-					number = i + 1,
-					url = href,
-					uploadDate = parseChapterDate(
-						dateFormat,
-						li.selectFirst("span.chapter-release-date i")?.text(),
-					),
-					source = source,
-					scanlator = null,
-					branch = null,
-				)
-			},
+			chapters = chaptersDeferred.await(),
 		)
 	}
 
+	protected open suspend fun getChapters(manga: Manga, doc: Document): List<MangaChapter> {
+		val root2 = doc.body().selectFirstOrThrow("div.content-area")
+			.selectFirstOrThrow("div.c-page")
+		val dateFormat = SimpleDateFormat(datePattern, sourceLocale ?: Locale.US)
+		return root2.select("li").asReversed().mapChapters { i, li ->
+			val a = li.selectFirst("a")
+			val href = a?.attrAsRelativeUrlOrNull("href") ?: li.parseFailed("Link is missing")
+			MangaChapter(
+				id = generateUid(href),
+				name = a.ownText(),
+				number = i + 1,
+				url = href,
+				uploadDate = parseChapterDate(
+					dateFormat,
+					li.selectFirst("span.chapter-release-date i")?.text(),
+				),
+				source = source,
+				scanlator = null,
+				branch = null,
+			)
+		}
+	}
+
 	override suspend fun getPages(chapter: MangaChapter): List<MangaPage> {
 		val fullUrl = chapter.url.toAbsoluteUrl(getDomain())
 		val doc = context.httpGet(fullUrl).parseHtml()
@@ -180,7 +190,7 @@ internal abstract class MadaraParser(
 		}
 	}
 
-	private fun parseChapterDate(dateFormat: DateFormat, date: String?): Long {
+	protected fun parseChapterDate(dateFormat: DateFormat, date: String?): Long {
 		date ?: return 0
 		return when {
 			date.endsWith(" ago", ignoreCase = true) -> {
@@ -295,10 +305,13 @@ internal abstract class MadaraParser(
 	@MangaSourceParser("MANGAREAD", "MangaRead", "en")
 	class MangaRead(context: MangaLoaderContext) : MadaraParser(context, MangaSource.MANGAREAD, "www.mangaread.org") {
 		override val tagPrefix = "genres/"
+		override val datePattern = "dd.MM.yyyy"
 	}
 
 	@MangaSourceParser("MANGAWEEBS", "MangaWeebs", "en")
-	class MangaWeebs(context: MangaLoaderContext) : MadaraParser(context, MangaSource.MANGAWEEBS, "mangaweebs.in")
+	class MangaWeebs(context: MangaLoaderContext) : MadaraParser(context, MangaSource.MANGAWEEBS, "mangaweebs.in") {
+		override val datePattern = "dd MMMM HH:mm"
+	}
 
 	@MangaSourceParser("KINGMANGA", "KingManga", "en")
 	class KingManga(context: MangaLoaderContext) : MadaraParser(context, MangaSource.KINGMANGA, "king-manga.com") {
@@ -308,6 +321,7 @@ internal abstract class MadaraParser(
 		}
 	}
 
+	@Deprecated("Not working, unknown reason, maybe shutdown or domain expired")
 	@MangaSourceParser("MANGAHATACHI", "MangahaTachi", "ja")
 	class MangahaTachi(context: MangaLoaderContext) :
 		MadaraParser(context, MangaSource.MANGAHATACHI, "mangahatachi.com")
@@ -316,7 +330,7 @@ internal abstract class MadaraParser(
 	class PianManga(context: MangaLoaderContext) : MadaraParser(context, MangaSource.PIANMANGA, "pianmanga.com")
 
 	@MangaSourceParser("MANGAROSIE", "MangaRosie", "en")
-	class MangaRosie(context: MangaLoaderContext) : MadaraParser(context, MangaSource.MANGAROSIE, "mangarosie.me")
+	class MangaRosie(context: MangaLoaderContext) : MadaraParser(context, MangaSource.MANGAROSIE, "mangarosie.love")
 
 	@MangaSourceParser("READMANWHA", "ReadManwha", "en")
 	class ReadManwha(context: MangaLoaderContext) : MadaraParser(context, MangaSource.READMANWHA, "readmanwha.net")
@@ -325,7 +339,9 @@ internal abstract class MadaraParser(
 	class MangaTx(context: MangaLoaderContext) : MadaraParser(context, MangaSource.MANGATX, "mangatx.com")
 
 	@MangaSourceParser("MANGAEFFECT", "MangaEffect", "en")
-	class MangaEffect(context: MangaLoaderContext) : MadaraParser(context, MangaSource.MANGAEFFECT, "mangaeffect.com")
+	class MangaEffect(context: MangaLoaderContext) : MadaraParser(context, MangaSource.MANGAEFFECT, "mangaeffect.com") {
+		override val datePattern = "dd.MM.yyyy"
+	}
 
 	@MangaSourceParser("AQUAMANGA", "AquaManga", "en")
 	class AquaManga(context: MangaLoaderContext) : MadaraParser(context, MangaSource.AQUAMANGA, "aquamanga.com") {
@@ -347,7 +363,9 @@ internal abstract class MadaraParser(
 	}
 
 	@MangaSourceParser("HARIMANGA", "HariManga", "en")
-	class HariManga(context: MangaLoaderContext) : MadaraParser(context, MangaSource.HARIMANGA, "harimanga.com")
+	class HariManga(context: MangaLoaderContext) : MadaraParser(context, MangaSource.HARIMANGA, "harimanga.com") {
+		override val datePattern = "MM/dd/yyyy"
+	}
 
 	@MangaSourceParser("KISSMANGA", "KissManga", "en")
 	class KissManga(context: MangaLoaderContext) : MadaraParser(context, MangaSource.KISSMANGA, "kissmanga.in")
@@ -355,9 +373,6 @@ internal abstract class MadaraParser(
 	@MangaSourceParser("MANGAROCK", "MangaRock", "en")
 	class MangaRock(context: MangaLoaderContext) : MadaraParser(context, MangaSource.MANGAROCK, "mangarockteam.com")
 
-	@MangaSourceParser("ISEKAISCAN_EU", "IsekaiScan (eu)", "en")
-	class IsekaiScanEu(context: MangaLoaderContext) : MadaraParser(context, MangaSource.ISEKAISCAN_EU, "isekaiscan.eu")
-
 	@MangaSourceParser("FREEMANGA", "FreeManga", "en")
 	class FreeManga(context: MangaLoaderContext) : MadaraParser(context, MangaSource.FREEMANGA, "freemanga.me")
 
@@ -385,7 +400,9 @@ internal abstract class MadaraParser(
 	class ManhwaClan(context: MangaLoaderContext) : MadaraParser(context, MangaSource.MANHWACLAN, "manhwaclan.com")
 
 	@MangaSourceParser("MANGA_3S", "Manga3s", "en")
-	class Manga3s(context: MangaLoaderContext) : MadaraParser(context, MangaSource.MANGA_3S, "manga3s.com")
+	class Manga3s(context: MangaLoaderContext) : MadaraParser(context, MangaSource.MANGA_3S, "manga3s.com") {
+		override val tagPrefix = "manhwa-genre/"
+	}
 
 	@MangaSourceParser("MANHWAKOOL", "Manhwa Kool", "en")
 	class ManhwaKool(context: MangaLoaderContext) : MadaraParser(context, MangaSource.MANHWAKOOL, "manhwakool.com") {
@@ -397,6 +414,7 @@ internal abstract class MadaraParser(
 	@MangaSourceParser("TOPMANHUA", "Top Manhua", "en")
 	class TopManhua(context: MangaLoaderContext) : MadaraParser(context, MangaSource.TOPMANHUA, "www.topmanhua.com") {
 		override val tagPrefix = "manhua-genre/"
+		override val datePattern = "MM/dd/yyyy"
 	}
 
 	@MangaSourceParser("X2MANGA", "X2Manga", "en")
@@ -491,6 +509,8 @@ internal abstract class MadaraParser(
 	@MangaSourceParser("TREE_MANGA", "Tree Manga", "en")
 	class TreeManga(context: MangaLoaderContext) : MadaraParser(context, MangaSource.TREE_MANGA, "treemanga.com") {
 
+		override val datePattern = "MM/dd/yyyy"
+
 		override fun getFaviconUrl(): String {
 			return "https://${getDomain()}/wp-content/uploads/2017/10/lgoo-treemanga-2-1.jpg"
 		}
@@ -525,7 +545,9 @@ internal abstract class MadaraParser(
 	class Manga365(context: MangaLoaderContext) : MadaraParser(context, MangaSource.MANGA_365, "365manga.com")
 
 	@MangaSourceParser("MANGACLASH", "Mangaclash", "en")
-	class Mangaclash(context: MangaLoaderContext) : MadaraParser(context, MangaSource.MANGACLASH, "mangaclash.com")
+	class Mangaclash(context: MangaLoaderContext) : MadaraParser(context, MangaSource.MANGACLASH, "mangaclash.com") {
+		override val datePattern = "MM/dd/yyyy"
+	}
 
 	@MangaSourceParser("ZINMANGA", "ZINMANGA", "en")
 	class Zinmanga(context: MangaLoaderContext) : MadaraParser(context, MangaSource.ZINMANGA, "zinmanga.com")

src/main/kotlin/org/koitharu/kotatsu/parsers/site/madara/MangaTrendParser.kt

@@ -0,0 +1,51 @@
+package org.koitharu.kotatsu.parsers.site.madara
+
+import org.jsoup.nodes.Element
+import org.koitharu.kotatsu.parsers.MangaLoaderContext
+import org.koitharu.kotatsu.parsers.MangaSourceParser
+import org.koitharu.kotatsu.parsers.model.Manga
+import org.koitharu.kotatsu.parsers.model.MangaChapter
+import org.koitharu.kotatsu.parsers.model.MangaSource
+import org.koitharu.kotatsu.parsers.model.MangaState
+import org.koitharu.kotatsu.parsers.util.attrAsAbsoluteUrlOrNull
+import org.koitharu.kotatsu.parsers.util.mapToSet
+import org.koitharu.kotatsu.parsers.util.selectFirstOrThrow
+
+@MangaSourceParser("MANGATREND", "MangaTrend", "ar")
+internal class MangaTrendParser(context: MangaLoaderContext) :
+	Madara6Parser(context, MangaSource.MANGATREND, "mangaatrend.com") {
+
+	override val tagPrefix = "manga-genre/"
+
+	override fun getFaviconUrl(): String {
+		return "https://${getDomain()}/wp-content/uploads/2022/08/23UOmUDN_400x400-150x150.png"
+	}
+
+	override fun parseDetails(manga: Manga, body: Element, chapters: List<MangaChapter>): Manga {
+		val root = body.selectFirstOrThrow(".site-content")
+		val postContent = root.selectFirstOrThrow(".post-content")
+		val tags = postContent.getElementsContainingOwnText("التصنيفات")
+			.firstOrNull()?.tableValue()
+			?.getElementsByAttributeValueContaining("href", tagPrefix)
+			?.mapToSet { a -> a.asMangaTag() } ?: manga.tags
+		return manga.copy(
+			largeCoverUrl = root.selectFirst("summary_image")
+				?.selectFirst("img[src]")
+				?.attrAsAbsoluteUrlOrNull("src"),
+			description = (root.selectFirst(".detail-content")
+				?: root.selectFirstOrThrow(".manga-excerpt")).html(),
+			author = postContent.getElementsContainingOwnText("المؤلف")
+				.firstOrNull()?.tableValue()?.text()?.trim(),
+			state = postContent.getElementsContainingOwnText("الحالة")
+				.firstOrNull()?.tableValue()?.text()?.asMangaState(),
+			tags = tags,
+			isNsfw = body.hasClass("adult-content"),
+			chapters = chapters,
+		)
+	}
+
+	override fun String.asMangaState() = when (trim()) {
+		"OnGoing" -> MangaState.ONGOING
+		else -> null
+	}
+}

src/main/kotlin/org/koitharu/kotatsu/parsers/util/Assert.kt

@@ -0,0 +1,8 @@
+package org.koitharu.kotatsu.parsers.util
+
+fun <T : Any> T?.assertNotNull(name: String): T? {
+	assert(this != null) {
+		"Value $name is null"
+	}
+	return this
+}

src/main/kotlin/org/koitharu/kotatsu/parsers/util/Parse.kt

@@ -68,10 +68,13 @@ fun String.toAbsoluteUrl(domain: String): String = when {
 }
 
 fun DateFormat.tryParse(str: String?): Long = if (str.isNullOrEmpty()) {
+	assert(false) { "Date string is null or empty" }
 	0L
 } else {
 	runCatching {
 		parse(str)?.time ?: 0L
+	}.onFailure {
+		assert(false) { "Cannot parse date $str: ${it.message}" }
 	}.getOrDefault(0L)
 }
 

src/main/kotlin/org/koitharu/kotatsu/parsers/util/json/JsonExt.kt

@@ -136,4 +136,8 @@ fun JSONArray?.isNullOrEmpty(): Boolean {
 	}
 
 	return this == null || this.length() == 0
+}
+
+fun JSONArray.toJSONList(): List<JSONObject> {
+	return List(length()) { i -> getJSONObject(i) }
 }

src/test/kotlin/org/koitharu/kotatsu/parsers/MangaParserTest.kt

@@ -38,7 +38,7 @@ internal class MangaParserTest {
 		val page2 = parser.getList(page1.size, sortOrder = null, tags = null)
 		val intersection = page1.intersect(page2.toSet())
 		assert(intersection.isEmpty()) {
-			"Pages are intersected: " + intersection.joinToString { it.publicUrl }
+			"Pages are intersected by " + intersection.size
 		}
 	}