Experiencing Rate Limits Due to Feed Updates #596

New Issue

2023-12-16T13:48:40Z

PhantomShift commented

2023-12-16 13:48:40 +00:00

(Migrated from github.com)

Brief summary

I am not entirely sure if this is a parser-specific issue, as I exclusively use Mangadex, but as far as I'm aware the relevant code is within this repository. Perhaps the main issue is the large number of manga I have in my history, but updating the feed seems to quickly run into Mangadex's rate limit and causes the client to no longer be able to fetch any data, making reading/updating impossible until the limit expires or my IP address changes.

I'm not very familiar with Kotatsu's code base as I've only inspected it lightly but I think that the relevant code which updates the feed doesn't impose a limit on itself (or that limit is still beyond Mangadex's limit of 5 requests per second) and just runs as many requests as possible until it finishes or an error occurs, most commonly a rate limit error, which may be related to why some manga appear to "never be updated" despite them having been updated whenever lots of manga are bookmarked.

I'm not sure if having lots of manga being updated in the feed was outside of expectations but having some sort of limit on the speed at which it updates would be greatly appreciated in order to avoid these problems in the future, assuming that is the problem.

Steps to reproduce

No response

Kotatsu version

6.4.2

Android version

13.0

Device

OnePlus 10T

Acknowledgements

I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open issue.
If this is an issue with a parser, I should be opening an issue in the parsers repository.

### Brief summary I am not entirely sure if this is a parser-specific issue, as I exclusively use Mangadex, but as far as I'm aware the relevant code is within this repository. Perhaps the main issue is the large number of manga I have in my history, but _updating the feed seems to quickly run into Mangadex's rate limit and causes the client to no longer be able to fetch any data_, making reading/updating impossible until the limit expires or my IP address changes. I'm not very familiar with Kotatsu's code base as I've only inspected it lightly but I think that the relevant code which updates the feed doesn't impose a limit on itself (or that limit is still beyond Mangadex's limit of 5 requests per second) and just runs as many requests as possible until it finishes or an error occurs, most commonly a rate limit error, which may be related to why [some manga appear to "never be updated"](https://github.com/KotatsuApp/Kotatsu/issues/516) despite them having been updated whenever lots of manga are bookmarked. I'm not sure if having lots of manga being updated in the feed was outside of expectations but having some sort of limit on the speed at which it updates would be greatly appreciated in order to avoid these problems in the future, assuming that is the problem. ### Steps to reproduce _No response_ ### Kotatsu version 6.4.2 ### Android version 13.0 ### Device OnePlus 10T ### Acknowledgements - [X] I have searched the existing issues and this is a new ticket, **NOT** a duplicate or related to another open issue. - [X] If this is an issue with a parser, I should be opening an issue in the [parsers repository](https://github.com/KotatsuApp/kotatsu-parsers/issues/new/choose).

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: KotatsuApp/Kotatsu#596