v0.5.5 - Performance Part 1
Published on August 17, 2022 by Joseph Milazzo
This is another release where I forgot just how much I did. The intent of this release was performance and held tons of work that I had been wanting to tackle, but needed the dedicated time for. Originally I had planned to release this with the new scan loop, however due to security vulnerabilities and some bugs in the hotfix, I had to split the release in half. As mentioned, the focus is on performance and we are bringing a ton of nice performance improvements along with a new contributor, @TheIceCreamTroll, who not only opened a lot of bugs on Github, but also helped fix a few in the process. I look forward to more development work from them.
To start off this highlight, comes Downloads. Downloads have always been pretty reliable but when you're downloading tons of content for an upcoming trip, it can first get annoying when you leave the page, you might loose the download. Also that downloads kept prompting you for confirmation when the files are large (to hopefully not eat up your data cap). So this release, I reworked downloads significantly. Downloads are now handled at a global level, range processing is enabled (this means you can stop/start if your browser supports it), downloads are cached and thus streamed to your browser (less memory overhead), the event widget handling has been tweaked a lot to make it much more streamlined, and my favorite, if you start a download, go to another page then return, the download progress bar on the card will still show.
Another huge update for Kavita is moving to OnPush change detection, something that is more technical and more work for me, but has nice performance implications for the user. OnPush basically means that the code tells Angular when it's time to re-render, which means less work Angular does and hence more frames per second. Pairing this with the update to Angular 14, the UI should feel a lot snappier to you.
This next one is my favorite. I was shocked when I was on a non-admin account and realized, I didn't have a way to track what I wanted to read...normally, I would create a collection but then I realized, Kavita's collections are for admin's only (modeled after Plex originally). So I sought out to build a Want to Read list that is like a collection that tracks what you want to read. Right now, we have ability to filter against it and that's it. I plan to build it out in future updates; if you have any ideas, jump on the feature request site and submit them.
Lastly, a few users have been having issues with the invite user flow. I've been hard at work trying to support these users, but have been unable to reproduce their issues. I made some changes in the Invite User flow to hopefully improve their lives, but behind the scenes I'm still engaged in support and planning enhancements around this Authentication system. Thank you for being patient. Early reports from users seem to have less issues post these changes, so give it another go if you fell into this bucket.
Lastly is a call to the community if anyone is interested in helping in Support in our discord or being a dedicated release tester. Kavita has gotten to the size where it is taxing to be building and testing all by myself. Currently release testing takes 3 days split between 3 people. I really need help from the community to help keep Kavita stable and let me focus on development rather than Support and Testing. If anyone is interested, please reach out to me in the discord.
With all that said, the next release will be even slower than this one. I have a lot of vacation planned for next 2 months and the left over work is extremely difficult, as it's the most complicated and critical part of Kavita. Thank you as always for being patient with releases and for those that have donated, I am extremely grateful. I've been buying my Kavita supplies (lots of coffee to get me through the scan loop).
Note: The security vulnerabilities may not be viewable till a few days after this release, to allow all platforms to upgrade their builds. Anything listed has been validated and fixed as of this release.
The full release can be found here.
To start off this highlight, comes Downloads. Downloads have always been pretty reliable but when you're downloading tons of content for an upcoming trip, it can first get annoying when you leave the page, you might loose the download. Also that downloads kept prompting you for confirmation when the files are large (to hopefully not eat up your data cap). So this release, I reworked downloads significantly. Downloads are now handled at a global level, range processing is enabled (this means you can stop/start if your browser supports it), downloads are cached and thus streamed to your browser (less memory overhead), the event widget handling has been tweaked a lot to make it much more streamlined, and my favorite, if you start a download, go to another page then return, the download progress bar on the card will still show.
Another huge update for Kavita is moving to OnPush change detection, something that is more technical and more work for me, but has nice performance implications for the user. OnPush basically means that the code tells Angular when it's time to re-render, which means less work Angular does and hence more frames per second. Pairing this with the update to Angular 14, the UI should feel a lot snappier to you.
This next one is my favorite. I was shocked when I was on a non-admin account and realized, I didn't have a way to track what I wanted to read...normally, I would create a collection but then I realized, Kavita's collections are for admin's only (modeled after Plex originally). So I sought out to build a Want to Read list that is like a collection that tracks what you want to read. Right now, we have ability to filter against it and that's it. I plan to build it out in future updates; if you have any ideas, jump on the feature request site and submit them.
Lastly, a few users have been having issues with the invite user flow. I've been hard at work trying to support these users, but have been unable to reproduce their issues. I made some changes in the Invite User flow to hopefully improve their lives, but behind the scenes I'm still engaged in support and planning enhancements around this Authentication system. Thank you for being patient. Early reports from users seem to have less issues post these changes, so give it another go if you fell into this bucket.
Lastly is a call to the community if anyone is interested in helping in Support in our discord or being a dedicated release tester. Kavita has gotten to the size where it is taxing to be building and testing all by myself. Currently release testing takes 3 days split between 3 people. I really need help from the community to help keep Kavita stable and let me focus on development rather than Support and Testing. If anyone is interested, please reach out to me in the discord.
With all that said, the next release will be even slower than this one. I have a lot of vacation planned for next 2 months and the left over work is extremely difficult, as it's the most complicated and critical part of Kavita. Thank you as always for being patient with releases and for those that have donated, I am extremely grateful. I've been buying my Kavita supplies (lots of coffee to get me through the scan loop).
Note: The security vulnerabilities may not be viewable till a few days after this release, to allow all platforms to upgrade their builds. Anything listed has been validated and fixed as of this release.
The full release can be found here.
Added
- Added new event hooks for when mark as read/unread occur on entities, which will now trigger a re-calculation on Read Time Left.
- Cover Image uploading will now allow gif and webp files
- When the user is requesting background jobs, like scan library, Kavita will now disregard jobs if there is already a job in the queue for that configuration.
- Downloads now support range processing (the browser can theoretically resume the download)
- Downloads will now persist through page loads and the indicator will show on any card/list items where applicable.
- Downloads are now global and a notification in event widget helps you see the download progress (browser progress).
- New user preference to prompt for downloads larger than 100MB. When disabled, Kavita will download without prompting.
- Downloads will now temp be left on disk and thus if requested again, can send the same file without re-generating it.
- New placeholder for when covers haven't loaded in yet.
- PDF Reader now has a loading indicator with a progress bar to indicate how much of the PDF has been downloaded by the browser.
- PDF Reader will now close itself when ESC is pressed
- Added the ability to manually setup users without having to worry about the invite url during invite flow or doing it right at invite time. Now a Setup button will appear next to all Pending invites and at any time can be pressed to manually complete the account setup. Note: This does not work with pending invites created before this update. Please re-create them to get the new functionality.
- Added back to top support on all pages but those that utilize virtual scrolling without a parent scroll
- Pressing H will open the shortcut modal
- All users will now have a dedicated List called 'Want to Read' which will allow them to track series they are interested in reading. They can apply filtering against this page.
- Added a quick way to set darkness level on manga reader for when night light just isn't dark enough (this is not a setting, it is being trailed and subject to removal)
- Added Japanese Series name parsing support in Parser
- Added Jump bar to reading list page
Changed
- Changed padding to allow for two cards side-by-side on viewport width of 375px.
- Changed every component to use OnPush Change Detection, resulting in less wasted render work and a snappier experience overall
- Moved some calculations for title and subtitle in manga reader to the backend
- Bulk Operations bar now has an explicit mark as read/unread buttons.
- Bulk Operations now has Add to Collection option and Delete where appropriate (and for appropriate users)
- Edit Series Relation will focus on the next typeahead after pressing Add Relation
- Cover Image chooser will now show the chapter's cover images as options by default
- Added more aggressive static file caching
- Refactored reading list item component to drastically reduce the amount or render calls. Huge performance for users with large reading lists
- Typeahead control now has an animation for opening and closing
- Scan Series, when invoked via UI/API, will force a re-calculation of metadata and ignore any caching, to ensure reliable results.
- Added Range processing support for images on the reader for slower networks or large files
- (Manga Reader) When using single mode, try to use a pre-fetched Image, rather than relying on browser to cache the image
- Reduced some latency when rendering first page of next chapter via continuous reading mode
- Downloads will now leave the file on disk and re-use them. These files are cleaned up nightly or after any scan (scan series/library)
- When a user is downloading a file on Kavita, admin's received a notification about it. This notification is now reworded to imply that the server is processing the download. This notification will be removed in future version.
- Changed how caching works within Kavita. Cover Images are now properly cached for 1 min, Images for the reader are cached for 10 mins, and the pdf file (for new reader) is also 10 mins. This has a known issue where updating a cover image, the browser wont reflect the new image on Chrome based browsers. This is due to Chrome not respecting the cache-control spec. A hard refresh will fix it.
- (Performance) Applied SplitQuery to multiple queries throughout the codebase, speeding up many API calls.
- Exception middleware will now send the original error message to the UI rather than a generic 'Internal Server Error'.
- Error toasts now have Error in the title along with the status code
- Changed volume titles in list view to include volume number.
- Implemented a workaround for nginx users with BlockCommonExploits enabled, which would interfere with book image escaping done by Kavita when images had ../ in their path.
- Hide jumpbar on pages where there is no scroll
- Non-admins can no longer view file info on card detail drawer
- Typeaheads will no longer close after selecting an item
- If there is no collection or reading list cover image, hide the image placeholder
- Optimized bookmarking a page by reducing a DB trip
- Search bar now will only show clear button once a single character is typed
- When making bulk selections, clicking anywhere on the card will select it.
- When cancelling a bulk selection action, like Adding to Reading List or Collection, cancelling the flow will no longer deselect the cards.
- Updated to Angular 14
- Updated dependencies and bootstrap to match Angular 14 version
- When saving bookmarks, capture their date, so we can serve them up in the order they were bookmarked. This only applies to new bookmarks.
- Updated all but one API in collection control to admin only policy
- Updated some headers on Kavita to tighten security
- Tightened up cover upload to restrict more APIs to the admin
- Resetting your own password now requires you to pass your current password (if already authenticated). Admins can still change passwords for the user without this restriction.
- Removed an additional copy in the build script and changed a copy to use appsettings.json from build rather than development.json
- Save buttons on user and admin dashboards will only activate if a setting is changed, and will deactivate when clicked
- The save button on the password reset screen will only activate if 'New Password' and 'Confirm Password' match
- Centered user roles to 'Roles' in the admin dashboard
- Reordered Series and entity details to be more consistent and renamed 'Read Left' to 'Time Left'
- Reading list page now has a Read button which starts at beginning and Continue which continues from last reading progress position
Fixed
- Fixed an issue where the sidenav wasn't scrollable on mobile.
- Fixed a bug where manage library component wasn't invoking the trackBy function
- Edit Series Relation now shows Parent as an option when the series is in a child relationship. This option is non-selectable and the field is disabled.
- Collection detail's scroller was on the wrong element and didn't match other pages
- When initializing side nav, don't call an authorized call until after user has authenticated
- Fixed a bug in book reader where page anchors weren't properly calculating and thus table of contents wouldn't show the highlighted item
- Fixed an edge case in Next Chapter API where if the user was on Volume 1 Chapter 1 and next chapter was Volume 2 Chapter 0, Kavita would report no next chapter.
- Added word breaking for long filenames with no spaces on headers, titles, and lists which would cause overflow.
- Titles on many screens and in modals will now word break if they are too long.
- Fixed a bug where Scan Series didn't properly choose the highest directory closest to the library path for scanning, leading to missed files (ie lib/Series A/First Files/a.cbz, lib/Series A/Second Files/b.cbz -> b.cbz would be skipped due to code choosing lib/Series A/First Files/ rather than lib/Series A/ for scanning)
- Fixed a bug where metadata filter, after updating a typeahead, collapsing the filter area then re-opening, the filter would still be applied to page, but the typeahead wouldn't show the modification (correct number of tags)
- Fixed a bad redirect on auth guard which could freeze the browser due to infinite redirect loop
- Fixed a bug when downloading bookmarks from multiple series, the temp file would only use the first series id rather than a combination of all of them.
- Fixed a bug on bookmark reader, the reader on last page would throw some errors and not show No Next Chapter toast.
- Fixed an issue where cover update events would be sent to the UI before the DB was saved, thus looking like covers weren't generated
- Fixed a bug where list item for books that have 0.X series index wouldn't render on series detail.
- Fixed issue where action menu would be overlapped by the header nav.
- Styling issue which caused reading list items to be shorter than intended.
- Fixed an issue where modal could extend past users max viewport height.
- (Manga Reader) Fixed an issue where sometimes when loading the next page, the pagination area wouldn't be properly setup due to a missed render cycle
- Fixed a flash of white when refreshing the browser
- Fixed a bug where bookmarking when using Double (manga) wouldn't bookmark both pages
- Lots of fixes around when we choose to render double pages, when we decide to skip pages, etc. This should hopefully be feature complete now.
- When an error occurs when registering a new user, delete the user instead of rolling back the transaction.
- Fixed Sequence contains no elements exception on first run (in stats)
- When progress updates were processed by Volume cards, the progress bar could be out of sync. This now correctly updates.
- Fixed an issue where InstallId wouldn't properly convert and show on the UI.
- Fixed a bug where browser would inject a clear button onto our search bar, showing 2 clear buttons.
- Fixed an oversight where if there is no tag in ComicInfo after a chapter was updated with People or Genres, then the People/Genres would never be removed.
- Fixed an issue where pages could be skipped in single due to prefetching overriding our recently set image (Manga Reader)
- Fixed dark mode on pdf reader not undoing correctly due to a bug in ngx-extended-pdf-viewer
- Fixed a bug with new authentication where images and pdfs would fail to load (from v0.5.4.2 hotfix)
- Fixed a regression from hotfix where bookmarks were not able to load due to a missing [AllowAnonymous] (from v0.5.4.2 hotfix)
- Fixed a bug where some config folders wouldn't be created on first load
- Fixed a regression which broke Plugins (and Tachiyomi) from authenticating (from v0.5.4.2 hotfix)
- Fixed a bug where clicking on a Jump key wouldn't go exactly to the selected letter
- Fixed a bug where . wasn't being included in # JumpKey
- Fixed a bug where the add to reading list modal would overflow and make create button hidden
- Fixed some inconsistencies with clearing multiple series of bookmarks in one go by building a dedicated api
- Fixed a bad naming case for reading list items where it was just a volume showing just Chapter 0
- Fixed a bug where there could be duplicate scrollbars on pages with virtual scroller
- Fixed a bug where scan series, when generating new covers, wouldn't emit cover updates to the UI
- Fixed an issue where opening directory picker to change bookmark directory and hitting Share without changing anything, would blank out the bookmark directory field.
- Summaries wouldn't be blurred if they were less than 250 characters.
- Custom css themes wouldn't load after v0.4.5.2 Hotfix
- On fresh installs, check if the Server Settings table exists so we don't dump an error on startup (confusing a handful of users)
- Fixed a bug where UI was showing Chapter created date for files instead of file created. This fix introduced a new field so will not show for existing files.
Security
- https://huntr.dev/bounties/456c8060-580a-4806-b2d8-fe47ba7d22c0/
- https://huntr.dev/bounties/c74a0cd4-2948-4dd3-9ed5-b1f2424ad832/
- https://huntr.dev/bounties/33e7fe0f-b3a0-4723-a8f9-de79d86b345d/
Known Issues
- ARM x64 users may have issues with PDFs generating cover images. There is a workaround found thanks to @kleisauke, the author of NetVips, which we use to generate images in Kavita. You can find information on our wiki's FAQ or the original issue for a workaround.
- Typeaheads (autocompletes) can clear itself automatically when adding multiple times in one go.
- Continuous scroller doesn't trigger in webtoon mode when in fullscreen
- Changing cover images on Chromium-based browsers will not reflect. This is due to Chromium not respecting the Cache-Control header (spec) and hence requires a hard refresh from the user.