Enhancing support for audiobooks

[Neurrone]

Hi,

I would love to be able to use Beets to manage my audiobook collection. I'm writing a plugin to fetch book data from Audible and find that it works very well when the audiobook is a folder of mp3s, where each mp3 is a chapter of the book.

There are two main scenarios that beets doesn't yet support, and I'm wondering if there are workarounds for this or if this is a feature that could be added?

1. M4b files with embedded chapter markers: this is probablyy the most common case, storing it in a single file is convenient for consumption and for file copying speeds.

   Can Beets somehow recognize that the file actually contains tracks?

2. Audio that is not split by chapter. This could be a single mp3, a single m4b without embedded chapter markers or a folder of many mp3s. Or this could also be a folder with subfolders (e.g, disk 1, disk 2 etc).

   If possible, I would want Beets to somehow not care about it not being organized by chapter but still apply the correct metadata to everything in that book. Or have some way of managing files that don't have a one-one relationship with tracks.

[sampsyo]

Both of these seem super interesting, and it would be great to support them! Neither one, however, is really supported in beets currently—neither mapping multiple "tracks" into a single file nor mapping a single "track" to multiple files.

They remind me a bit of #136, which is really old, and for which demand has really fallen off in recent years—but the idea there also is that people want to rip a CD as a single, continuous audio file and then logically view it as multiple tracks in beets.

Anyway, these both seem like interesting but ambitious projects. They would require some significant internal reorganization. So it might be best to start by getting familiar with the data structures and database layer in beets to understand what the scope would be.

[Neurrone]

It feels like supporting the first case, single file being able to represent an entire album would be easier to implement / start with.

Where should I start to look and at a high level, what would need to be done to support this?

[Neurrone]

1. Phase 1:
   • Adding both new columns
   • The read track functions can be updated to add the new logic to return subsections of a file, but because the track = file correspondance is still there, it should be backwards compatible. So the main work here is figuring out how to return the requested audio data based on start / end.
2. Phase 2: in this phase, the user shouldn't notice any changes in behaviour if they are not importing files with chapter markers. We could also gate the new functionality behind a feature flag.
   • Extend the i/o layer to recognize chapter markers in mp3 and m4b files and present them as separate tracks.
   • Update the writing infrastructure to be aware of the file / track distinction.
     • What happens if the chapter markers don't match track data fetched? I'm wondering how Beets currently handle duration mismatches. In this case, we have an option to correct these chapter markers if desired
     • Decide how to write id3 data to the enclosing file
   • Update file deletion so that it only happens when all tracks that reference it are deleted
   • Emit file_* events and figure out what parameters should be passed into listeners
3. Phase 3: start making backwards incompatible changes:
   • Update plugin events, I think it may be better to rename item_* to track_*.
   • Update path functions

[sampsyo]

Neat! Phase 2 seems like the hard one, to be honest, and we might need to break it down even further. But I'm down to proceed with the design for Phase 1…

[Neurrone]

Great! Starting with something simpler will also help me learn the codebase.

Some ideas / questions for starting to work on phase 1.

1. How does Beets read files and audio metadata from disk? E.g, getting details like duration of the file? Am looking for the code that does this to figure out how to return segments of a file.

2. Any preferences on how the database migration should be performed? What I have in mind is the 2 new non-nullable columns for start and end duration.

   • I noticed that length is a float in seconds, I was thinking of using an integer to store the start and end duration in ms instead. I usually default to integers when I can to avoid dealing with the complexity of floats.
   • Would this be a good time to store file information in a separate files table? Or keep path as is for now.
   • Would you want to keep the length column, or remove it because it can be deduced from end - start?

[sampsyo]

All the responsibilities for item 1 belong to MediaFile, which wraps the low-level data maculation library Mutagen.

I think the questions about the database schema might require a somewhat longer conversation… maybe that deserves a new discussion thread of its own?

[wisp3rwind]

• This will break backwards compatibility with events that assume an item maps to a single file:

This will actually break plugins in all kinds of ways (hooks, import stages, plugin commands, ...); the expectation of a one-to-one mapping between Items and media files is probably very deeply ingrained in the beets ecosystem. I would expect that this can easily lead to data loss due to unsuspecting plugins. Thus, planning this feature should also involve a plan on how to release it. Some things to consider:

• Step major version number
• Communication with plugin authors and packagers
• Think about implementing a mechanism to detect plugins that don't explicitly support "Beets 2.0" (this would be essential in my opinion, and in general nice to have should we want to break plugins again).
• Since this will involve a huge number of changes, think about which preparatory parts of the implementation can already be merged and shipped before actually enabling 1-to-N mapping of items.
• Maybe use the opportunity to break plugin API even further and remove some legacy code and unfortunate design decisions.

This is not to say it can't be done, but it is a big project.

[Neurrone]

I've pushed a preliminary version of an Audible plugin.

I hope to have more time in a few weeks to investigate extending Beets to better support this use case.

[Neurrone]

On further thought, I realize I've been making the assumption of being able to get chapter metadata (i.e, details for each track) for all books, which is probably not going to be the case. I am sure that I can get info on the overall book though.

I'm wondering if its possible to have Beets somehow support cases where track-level metadata isn't available and there's only overall info on the album? What I'd want to have is:

1. Some way of having Beets match tracks that are in the same book, even if my plugin isn't able to provide metadata at the level of individual chapters. So for example, this means the number of tracks in the album is unknown. I would have details on the book, description, authors etc though.
2. Write info about the book to ID3 tags on each track.

Is this currently possible to achieve within the existing plugin system? If so, I think this may be an easier interim step to supporting audiobooks that aren't properly chapterized.

• If chapter-level metadata is available and if the number of tracks doesn't match, then assume the book has not been chapterized, and don't return track-level metadata
• Without track-level matching, it falls back to the scenario I described above where matching is only done based on the album.

[Neurrone]

Thanks so much, I'm not sure how I didn't realize this.

This has facilitated a massive breakthrough, I believe I can get Beets to work well with virtually all audiobooks now. So much so that I feel the complexity from having Beets support multiple tracks per file wouldn't be worth the effort, unless there are other use cases besides audiobooks.

The key insight is to tell a white lie to Beets when the audio being added isn't chapterized. For example, if the entire book is in one file, I would return a single track instead of however many chapters the book actually has.

I'm still polishing the code but will update here when its ready for wider consumption.

[sampsyo]

That's awesome to hear!

[Neurrone]

its ready for use, see https://github.com/Neurrone/beets-audible

Let me know if the instructions are unclear. I need to create a mod for the linuxserver Beets container to make setting it up with Docker easier.

I've tested this on a variety of books and so far, it has worked on all of them.

[sampsyo]

Awesome. Are you interested in adding a link to the docs, in our list of third-party plugins? https://github.com/beetbox/beets/blob/master/docs/plugins/index.rst#other-plugins

[Neurrone]

Yup, I'll do so either this weekend or next.

I'd like to rename the plugin to audiobooks instead of audible just in case it grows to support additional sources of metadata.

[Neurrone]

Some quick questions:

• Is there a way for me to associate additional files to the album besides the cover art? I notice that if I edit the library to change e.g, the title or author of the album, the audio and cover gets correctly moved to the new location but there are additional files I write to the directory on initial import that get left behind.
• When importing an already chapterized book, sometimes it tries to make corrections to the track metadata that would mistag a chapter to the wrong chapter. For example, trying to change chapter 5 to chapter 7 because the length appears similar. Is there a way for me to ignore this change and have it proceeding with the import, but at the same time, apply other types of metadata like album artist, comments etc?
• I notice via verbose logging that most of the time, many candidates are filtered out because Beets detects search results that are similar as being duplicates even though they're not in this case. Is there a way for me to disable this behaviour?

[sampsyo]

There is not currently a way to associate additional files. That is the longstanding "attachments" proposal (e.g., #591) that has proved more difficult than expected to implement. It is also the topic of beets-copyartifacts.

I also can't currently think of a way to partially apply a match—i.e., just its album/albumartist but not any track metadata. Sounds potentially useful!

The matcher shouldn't be filtering out similar-but-not-identical matches. It just uses the ID—is there any chance these matches have the same ID?

[Neurrone]

The matcher shouldn't be filtering out similar-but-not-identical matches. It just uses the ID—is there any chance these matches have the same ID?

Which ID are you referring to? Is it the musicbrainze album id? I've set that to None because that seemed to be the most reasonable value in this instance, likewise for this id for all the tracks.

[sampsyo]

Yes. Give that a shot, at least—it's a design flaw, which has also proven very difficult to resolve, that this is the ID used to deduplicate matches (cf #604).

[Neurrone]

Do you mean assigning a unique musicbrainz id? I do have the asin which is unique.

Another temporary solution I could think of is to disable duplicate detection if the id is None, as it doesn't make sense for it to be done in this situation.

[sampsyo]

Yes, exactly. Both of those would be good options.