Open Collective
Open Collective
Loading
v0.8.0 - Comic & Collection Overhaul and PDF Settings Finally!
Published on April 12, 2024 by Joseph Milazzo

Have you ever thought to yourself that you'd like to switch the foundation of your house from slab to pier and beam? Well, that's essentially what I've done with this release, and not only that, I had to do it without a single picture falling off the walls. This release comprises a whopping 48K lines of code added and 5.5K removed throughout 400 files.

In order for me to achieve my vision of Kavita being the best software out there for reading, sometimes sacrifices have to be made. When I started building Kavita, I had never thought of what it would be today nor the part it would take in my life. It was simply a tool for me to consume and collect Manga since Ubooquity didn't have support for rich metadata. As users came along, so did the scope. Some features were added with limited knowledge, like comics, and that lack of knowledge in the beginning became a problem for heavy comic collectors. In order to build towards my vision of being the best, I had to rewrite large portions of how Kavita functions, work closely with the community, and deliver the best product I could. I want to first and foremost thank @DieselTech, who has been a key component in this. He has been pushing me relentlessly over the past year and has had a major hand in shaping the features, working with hardcore collectors from other servers (Mylar, Komga, and the CBL initiative).

Now, let's get into the meat and potatoes. What is new in this release?

Comic Overhaul

In order to support these hardcore users, Kavita needs to align with the tools already in the space, primarily Mylar and CBL. Mylar serves as the collection tool, aligning with Comic Vine for standards of naming and tagging, while CBL provides a rich repository of reading lists to make navigating comics easier. One of the most important differences between the old "Comic" library and the new "Comic Vine" library is that the Volume number is always the year and is used in the generation of the Series title. The new library type is strictly for those who abide by the rules laid out by Comic Vine and CBL. CBL works around these assumptions for linking up issues from many series into one reading list.

But that's not all. Comics are unique; they can have Issue Ω, which should be between Issues 8 and 9, or how about issue 18.HU? These are unique situations that didn't particularly go well with Kavita's number-based system. Through a feat of engineering, Kavita can now support these situations and, not only that, will soon allow the admins to actually change the sort order, a feature which, to my knowledge, no other software supports. This also means that Issue 0 (which can happen in Manga too) is now supported.

From the massive amount of testing, users have reported that this new library meets their needs perfectly. There are other small things, like Annual Series relationships, updates to CBL Import flows, etc., but the major points are above. This feature took 30K lines of code and 3 weeks of development for me to deliver to the testers (plus 2 weeks more of testing).

Collection Overhaul

While we are on the topic of reworking major code, let's talk about collections. When I first coded these, I made a really bad call and tied them with Series (so a series belongs to a collection), but as I started working on new collection-related features, I realized how bad of a call this was. I needed collections to be user-based, like want-to-read or reading lists. While this isn't really needed now, it will be with a future update.

So, yet again, I had to plan on how to migrate user data to the new collections without data loss and rework a ton of code. These new collections are user-based, and through a new Role, non-admins can even promote their own collections and share them with the server. While I was in there, I also made bulk promote/demote easier. Another easy 5K lines of code added and just over 1K removed.

Misc

Finally, the large, foundational features are over, and we can talk about some of the smaller things, like finally adding PDF settings a year after adding a PDF reader. Yikes. That's right, I finally got around to it and added PDF settings, along with also adding tap-to-paginate to the PDF reader.

Another area that has been receiving attention is Themes. Kavita saw an influx of new themes recently (and a few bug fixes too). Quite a few users are using these themes, and they look pretty good. Check them out on our Theme Repo. It's really easy to build your own, so if you have an interest, I highly encourage you to give it a go.

Another thing you might have noticed is Kavita+ enhancements. Quite a few small things were updated, but the most important is that Kavita+ will now auto-fetch data when you add new series and slowly work in the background to fetch external metadata. I have more planned around reporting on the data and managing bad matches against Kavita+, especially as I am still planning on basic metadata fetching, but I have found that this drastically improves load times on new series and makes the experience much better.

Lastly, I mentioned this last release, but Diesel has been hard at work on a complete revamp of the wiki. As of v0.8, we have moved over to the new wiki. Older installs wiki links will no longer work. Thanks again to @DieselTech for yet another monumental add to Kavita. 


Important

  • Important! Once you update to this release there are manually steps needed! You MUST perform a forced library scan on all libraries to migrate properly to the new foundation. Failure to do so may result in data loss. 

  • This is a massive update, all progress data will be exported on the first run to config/progress_export.csv. The migrations may take up to 10 minutes to migrate. Do not stop midway, you will break your db and need to restore from a backup. 

  • Kavita+ users: As mentioned last release, you are expected to stay up to date - 2 releases. Old Kavita+ APIs will be shutdown shortly after v0.8. You can update to v0.7.14 as the minimum (Note: v0.7.14 is not available at launch due to CI/CD. Just update to v0.8). 

  • The default emailer was supposed to be turned off last release. I held it longer to give time. The emailer will be turned off after the v0.8 release.

  • This is a MASSIVE update, some data loss can be expected. I spent weeks trying to minimize any data loss and have provided an export. I apologize ahead of time if any occurs. If you are running into issues, do not hesitate to reach out to support on discord. 

  • Unfortunately, during deployment, the CI/CD pipeline had an issue and v0.7.14 is now a pre-v0.8 nightly. We are current investigating a way to restore the old image for prosperity sake. 

The full release can be found here.

Added

- Added: Added a new library type aimed at Light Novels. This will use card list layout by default (overriding user preferences) and is eligible for Kavita+ metadata/scrobbling. The regular book library no longer is Kavita+ eligible.
- Added: Added a label on Library settings modal to help admins know which libraries are eligible for Kavita+ scrobbling/metadata pull.
- Added: (Kavita+) Implemented a background prefetch task that will slowly refresh series from Kavita+ (reviews, recs, ratings).
- Added: (Kavita+) Kavita will now prefetch data from Kavita+ on new series. This and the prefetch background task is limited to 50 series per 12 hours. Loading on-demand will still work as normal and is not restricted.
- Added: Chapters now sort by SortOrder. SortOrder is generated from chapter number, but can be overridden (in a future update). This allows for custom sorting for issues that aren't purely numerical. SortOrder will handle cases like 19, 19.HU, 19.BD, 20 without any user intervention.
- Added: You can now have Chapter 0, -1, etc.
- Added: This fully supports chapters with non-numeric issue numbers, like Alpha, 19.HU, etc.
- Added: Added a new library type: Comic Vine which is aimed at users of Mylar or have well tagged libraries following Comic Vine. This library works different and can support multiple folders with the same series parallel to library root.
- Added: New Series Relationship of Annual.
- Added: Imprints are now read from ComicInfo, will display on the UI and can be filtered.
- Added: (Scanner) Added a new LowestFolderPath. This is the lowest path that contains all the series files and is used by ScanFolder/Series to avoid extra work.
- Added: Added support for Locations and Teams, including the filtering for them
- Added: There is a one-time progress export to config/progress_export.csv in case of data loss.
- Added: Added Sort order to be visible on chapter detail drawer.
- Added: (Kavita+) Added Mal Username/ClientId integration into Kavita for upcoming Kavita+ integrations.
- Added: Added the ability to view (your) or all users (if admin) progress for that given chapter via the chapter detail drawer.
- Added: (Kavita+) Added a pie chart to Server Stats that shows how many eligible series have metadata fetched from Kavita+. This pairs well with the background job that fills out your server over time. (This will be expanded upon in a future update)
- Added: (Parser) Added parser support for Thai language volume and chapters (Thanks @gozilla-paradise)
- Added: Finally added PDF Reader Settings
- Added: (PDF Reader) Added tap to paginate to the pdf reader.
- Added: Collections can now be created by non-admins
- Added: Collections can now be promoted by non-admins (given they have the Promote role).
- Added: Added bulk flows for Promotion and Deletion. If you select cards that are not owned by you, nothing will happen to those cards.
- Added: OPDS feeds will now send description information when applicable for underlying volume/chapters.

Changed

- Changed: (Kavita+) Removed Book library from Scrobbling/Metadata pull. Books being in Google Books was far and few in-between. This is a temporary solution until Hardcover support is implemented, which has good Book (and some Comic) support. If you have light novels, change to the LN library type. The scanner works the same.
- Changed: Allow users to send files to their devices without having host name set for Email settings
- Changed: (Theme) side-nav-color is now side-nav-text-color
- Changed: (Theme) Added carousel-btn-color (for carousel buttons) and badge-text-color (for badge text)
- Changed: When iOS users are trying to download files greater than 200MB, they will be given an additional warning that iOS may fail to download due to arbitrary limits each device has.
- Changed: Specials will now order by SP marker numbers, else fallback to natural sort
- Changed: Loose Leaf volumes are now encoded with the number -100000 instead of 0. This now means users can have a Volume 0 without issue.
- Changed: Search will now search against a chapter range and will also fallback if there is no title (comicinfo title) to the range.
- Changed: Clean up a log message when no files found, to ensure the admin checks the Library settings as well.
- Changed: Image Library is now more aligned to reported usage
- Changed: Changed chapter title format to Chapter {number} - {title} (assuming title exists), else Chapter {number}. For Books, nothing has changed.
- Changed: Series Detail metadata area will collapse automatically on desktop if there are too many tags.
- Changed: Comic/Comic Vine libraries will never show Storyline or Volumes (unless there are no issues/specials or multiple volumes exist)
- Changed: Minor changes to how PDFs parse to make them a tad more smart. Nothing major, will do the overhaul later.
- Changed: Trim strings from ComicInfo more aggressively to prevent weird parsing for things like Number.
- Changed: Optimized a lot of the scanner and cleaned up the code deeply.
- Changed: (Parser) Adjusted the special parsing for Annuals to account for 'Series Annual \d'
- Changed: Dependency and security updates
- Changed: Added a flag to the CBL import to use Comic Vine parsing for CBL Import (and set to true as all CBLs to my knowledge are using this new library).
- Changed: Kavita now handles non-float numbers from CBL Imports (including special cases for comics)
- Changed: (Kavita+) During library setting modal, changing type will now automatically change the Allow Scrobbling control, if the underlying library type is not Kavita+ applicable.
- Changed: Normalize all paths during the scanner to ensure everything is always the same.
- Changed: Added the csv files to the backup as those are used in manual migrations and likely good to have.
- Changed: (Kavita+) When a rate limit error comes in after the pre-check, retry after the wait window instead of waiting for the next scrobble window.
- Changed: Some epubs have incorrect metadata where the series is parsing with a volume. To help users, Kavita will override this on Light Novel library type to merge into the correct series.
- Changed: Added a bit of a loader when deleting a lot of series with bulk actions.
- Changed: Tweaked some wording about the changelog component to better indicate to nightly users that are not on the latest stable-based nightlies.
- Changed: When using Force Scan on library settings, automatically close the modal.
- Changed: (Scanner) Change pdf parser so if it's a library type book, we override chapters as chapter number is RARELY used and mainly results in false positives.
- Changed: (Scanner) Fixed up the parsing so that book parser will coerced when the series in the metadata is wrong but there is a volume tag in said series, it will override.
- Changed: Added a fallback to the series detail page to handle bad pdf parses (until the upcoming pdf parser rewrite comes) to avoid no cards showing.
- Changed: Turned down the amount of debug logging from the library watcher and moved them to trace.
- Changed: Added some code so that while the device is offline, the UI wont try to refresh JWT and likewise, when turned back online, a JWT Refresh will be called to hopefully extend the session. 
- Changed: Collections will show who created them under the card. For the time being, actionables (...) will show for all cards. This system will be getting an overhaul in the future. You still wont be able to interact with another user's collections.
- Changed: Changed some of the format icons to some better looking icons
- Changed: Added a filter in Edit Collections > Series tab to quickly find series to remove in large collections
- Changed: Updated all email templates to have rel noopener noreferrer
- Changed: When checking Kavita+ license, show a loading spinner first.
- Changed: Reduced amount of Kavita+ license checks
- Changed: Send to Device now works without Hostname set.
- Changed: Switched all links over to the new wiki

Fixed

- Fixed: Fixed a security bug from .NET (CVE-2024-21409)
- Fixed: Fixed an issue around being able to delete series with external recommendations.
- Fixed: Fixed up the delete series flow so that the boolean returned is used in the UI to pop the correct toaster.
- Fixed: Fixed an oversight where emailers without authentication wouldn't work
- Fixed: Fixed a bug with looping around to volume 1 at the end of a reading session when it shouldn't 
- Fixed: Fixed a case where an exception was thrown, scanner wasn't existing and instead invoked metadata service with bad entities
- Fixed: Fixed some bad styling on Chapter metadata (drawer)
- Fixed: Fixed a bad localization on edit reading list modal.
- Fixed: Fixed another check around rescheduling Kavita+ tasks once a valid check came through. There was a lack of a checking mechanism and Kavita was doing more than needed.
- Fixed: I believe the Foreign Key constraint issue is solved once and for all 
- Fixed: Fixed a rendering bug on CBL validation where the UI reports success but also a warning message appears
- Fixed: Fixed a bug where Translators couldn't be updated in Edit Series Modal.
- Fixed: Fixed a manual migration that had to run before the db was migrated and it didn't check if the db even existed yet.
- Fixed: Scanner was clearing temp directory when it shouldn't have been, which could replace db migration backup.
- Fixed: Added an argument exception when reording a list has a number that's less than 0, throw. This usually happens with bad metadata in ComicInfo's. Perhaps when there is a -1 chapter, it's using number instead of order.
- Fixed: Stats and Tasks were missing localization 
- Fixed: Fixed an issue where the new downloading code would unzip epubs on volume/series download 
- Fixed: Fixed a bug where opening library settings from ... on library detail page wouldn't have library type. 
- Fixed: Fixed a bug where the UI wasn't properly sending the custom cron data to the backend. 
- Fixed: Fixed a bug where weblink's in a card detail drawer wasn't rendering the favicon
- Fixed: Fixed an issue where progress bar css variable was being overridden by bootstrap
- Fixed: Fixed a bug where selecting bulk mode on in customize side nav wouldn't allow you to ever enable order numbers again.
- Fixed: (Parser) Fixed a parsing case for 'Cynthia The Mission - c000 - c006 (v06)'
- Fixed: Fixed an issue where uploading a custom cover wasn't respecting Cover Size setting (Thanks Elry)
- Fixed: Fixed a bug where an epub with an empty link element could break Kavita's epub reader (Thanks @YodaDaCoda)
- Fixed: (Kavita+) Fixed External reviews not showing the Go to Review due to an API change.
- Fixed: Fixed an issue where applying a custom theme was not working immediately or consistently
- Fixed: Fixed bad messaging around test email failing
- Fixed: Fixed an issue when changing email address, after validating, it wouldn't mark as confirmed.
- Fixed: Panels wasn't properly showing images on Series due to a bug on their app (Thanks Dani)
- Fixed: Panels wasn't properly syncing read status with Kavita (Thanks Dani)
- Fixed: Avoid possible username enumeration in login or reset-password endpoints due to message translations (Thanks @YodaDaCoda)
- Fixed: Ensure we cannot create multiple collections in bulk flow if we use ENTER Key 
- Fixed: When clicking Gmail/Outlook buttons on email form, the form wasn't allowing saving even though fields changed.
- Fixed: (Panels) Panels has fixed a few bugs with Kavita. v3.4.1 (beta) and above should be used

Removed

- Removed: Removed .kavitaignore file support as Library Exclude patterns are much easier to use and flexible enough.

API

- NOTE CDisplayEx, Panels, Tachiyomi, and Komf have all been tested with v0.8. Aidoku and Kavya (Paperback) are in progress. 
- VolumeDto reverted the float on Number field back to int
- ChapterDto Range will be the chapter number. If a special, will be the special name. MinNumber can always be checked for the special encoding
- Encoding: Special (100000), Loose Leaf Volume (-100000).
- stats/user/reading-history has a dto change, making ChapterNumber a float instead of a string.
- Tachiyomi routes now use their own Dto to avoid having to break apis there.
- Added SortOrder on the ChapterDto. This is a float that can be used to order the chapters within the Volume. By default, all APIs will already order by this.
- SeriesMetadataDto (for updating series metadata) now has Teams, Locations, Imprints and their lock fields: teamLocked, locationLocked, imprintLocked.
- Changed library/ -> library/libraries for getting all libraries
- All Collection apis were updated, however most of the underlying DTO is the same.
- SeriesMetadata DTO no longer has Collections or needs them for the POST (as they are no longer tied with this entity)
- series/ (deprecated) no longer handles filtering for collections. Use v2 api.