style-light-choice
style-dark-choice 
Avatar
Gary
Vollink.com
22 November 2019
Updated: 20 December 2019

Apple Music

Contents

Introduction
musicdb File Format
Envelope Header
Crypt Size
Encrypted AND zlib Compressed Payload
Inner Library Sections
hsma Section Boundary
hfma Outer Envelope (partial copy)
plma Library Master
lama iama Master (Album list?)
iama iama Item (Albums?)
lAma iAma Master (Artist list?)
iAma iAma Item (Artists?)
ltma Tracks/Items Master
itma Track/Item Container
lPma Playlists Master
lpma Playlist Container
boma (partial) Associated Data Container
boma SubType 0x0001 Related Numeric Values
boma Widechar SubTypes Related Wide Character Values
boma Character SubTypes Related XML Blocks
boma book SubTypes Related Strings
boma ipfa SubType 00CE Playlist Track
boma Unknown Unknown
ssma EOF Marker (not always present)
Want To Help?
Acknowledgements

Introduction

In macOS Catalina, Apple has replaced iTunes with individual applications called Books, TV, Podcasts and - the subject of this exploration - Apple Music.  Importantly, Apple Music no longer has an option to automatically keep an xml copy of the library available for third party tools (like Playlister).

This is a sub-project of Playlister, and is likely to become a stand-alone library that Playlister will be able to use if someone's music library is now managed by Apple Music under macOS Catalina.  As this is a hobby project, I do not expect to be finished quickly. 

Upon first run, Apple Music imports the original iTunes Library.itl file, and it also seemed to modify that file, though I don't have the ability to re-test to verify this.  After initial import, Apple Music does not open or do anything further with the iTunes Library.itl file.

Apple Music creates a folder called Music (inside $HOME/Music), and inside that creates a sparsebundle:
    /Users/gary/Music/Music/Music Library.musiclibrary

Inside Music Libary.musiclibrary are seven files.  Of these, three files have the extension, musicdb:
  Application.musicdb
  Library Preferences.musicdb
  Library.musicdb

musicdb File Format

I'm documenting the file format of the new musicdb format as I figure it out.  Unlike the iTunes itl format, all numbers (including IDs) are little endian, meaning they can be natively read by Intel/AMD x86 CPUs.  Large portions of this format are a mystery to me, and probably will continue to be.

I started this exploration with the Library.musicdb, because it has the most promising name, and is the largest of these files.  So far, all of the information I would expect to find is in this file, so I have put the most effort into making the documentation follow that file's format. 

The sections listed in the Table of Contents above are in the approximate order of the first appearance of each of those file sections (or problems to solve) in the Library.musicdb file.  Those sections that are listed but not documented can be, as safely as possible, skipped. 

While not complete, I believe that most of the information that could be pulled from the iTunes xml format can be recreated from the data-points documented below.

Envelope Header

OffsetLength InformationValue (example)
(0)4 File signature hfma
(4)4 Envelope length in bytes A0 00 00 00 (160)
(8)4 File length in bytes 87 16 00 00 (5767)
(12)4 Unknown 01 00 00 00
(16)32 Version of Apple Music, NULL terminated string. 1.0.1.37\0
(48)8 Library Persistent ID, 64bit Value F60BBBD97C7D8F41
(56)4 Unknown 06 00 00 00
...4 ... ...
(88)4 Unknown B0 B9 FF FF
(92)4 Max Crypt Size (See crypt size formula) 71 0B 8E 07 (126749553)
(96)4 Unknown 00 00 00 00
(100)4 Library Date * 0F CE 0C DA (3658272271) *2019-12-04T02:44:31Z
(104)4 Unknown 00 00 00 00
...4 ... ...

* Dates are expressed in Seconds since 1-Jan-1904 Midnight.

Crypt Size

If Max Crypt Size is smaller than the File size, use it directly.

Crypt Size = File Size - Envelope Length - ((File Size - Envelope Length) mod 16)

Encrypted AND zlib Compressed Payload

Starting with the byte offset (Envelope length in Bytes), Crypt Size of the file needs to be decrypted using the standard AES128-ECB algorythm.  OpenSSL and macOS built-in libraries can do this in C/C++.  In Perl, use Crypt::Cypher.  A key is necessary to successfully unencrypt this. 

Once unencrypted, the payload must be expanded/inflated via zlib, including the rest of the file (if any) larger than the initial Max Crypt Size.

Back to Table of Contents.

Inner Library Sections

Almost all* sections follow this beginning pattern, with differences following these first 8 bytes.

OffsetLength InformationValue (example)
(0)4 Section signature xxxx
(4)4 Next Section Offset 38 00 (56)

These section headers follow the base pattern as described above:
hsma, hfma, plma, lama, iama, lAma, iAma, ltma, itma, lPma, and lpma.

* boma and ssma sections do not follow this pattern.

Back to Table of Contents.

hsma Section Boundary

Occurs multiple times (6).  It marks transitions between major sections of the file. 

OffsetLength InformationValue (example)
(0)4 Section signature hsma
(4)4 Next Section Offset 38 00 00 00 (56)
(8)4 Associated Sections Length D8 00 00 00 (216)
(12)4 Section SubType 03 00 00 00 (3)
...4 Zeroes 00 00 00 00 (0)

Section SubTypes are
3 - First Boundary (first encrypted/compressed area), holds inner hfma.
6 - Holds Library Master (plma) and associated (boma) data.
4 - Holds lama and associated data.
5 - Holds lAma and associated data.
1 - Holds Track Master (ltma) and associated data.
3 - Holds Playlist Master (lPma) and associated data.

Associated Sections Length is counted from the beginning of this hsma section, and will point to either the next hsma section or the end of the file.

Back to Table of Contents.

hfma

Many parts of this are a repeat of the Outer Envelope hfma.

OffsetLength InformationValue (example)
(0)4 Section signature hfma
(4)4 Section length in bytes A0 00 (160)
(8)4 Unknown 00 00 00 00 (0)
(12)4 Unknown 01 00 00 00 (1)
(16)32 Version of Apple Music, NULL terminated string. 1.0.1.37\0
(48)8 Library Persistent ID, 64bit Value F60BBBD97C7D8F41
(56)4 Unknown 06 00 00 00 (6)
...4 ... ...
(88)4 Unknown B0 B9 FF FF
(92)4 Max Crypt Size (See crypt size formula) 07 8E 0B 71 (126749553)
(96)4 Unknown 00 00 00 00
(100)4 Library Date 0F CE 0C DA (3658272271) 2019-12-04T02:44:31Z
(104)4 Unknown 00 00 00 00
(108)8 Library Persistent ID 2 F60BBBD97C7D8F41
...4 ... ...

Back to Table of Contents.

plma Library Master

This appears to be the Library Master.  It occurs only once.  This section is followed by boma sections, the number of which is indicated at offset 8. 

OffsetLength InformationValue (example)
(0)4 Section signature plma
(4)4 Section length in bytes A0 00 (160)
(8)4 How Many boma sections follow 06 00 00 00 (6)
...4 Unknown ...
(58)8 Library Persistent ID, 64bit Value F60BBBD97C7D8F41
...4 Unknown ...
(92)8 Library Persistent ID, 64bit Value F60BBBD97C7D8F41
...4 Unknown ...

Back to Table of Contents.

lama Master for iama

Appears only once.  Contains a count for iama sections in the library.  May be a list of Albums/Collections.

OffsetLength InformationValue (example)
(0)4 Section signature lama
(4)4 Section length in bytes 30 00 00 00 (48)
(8)4 How Many iama sections follow 03 00 00 00 (3)
...4 Zeroes 00 00 00 00 (0)

Back to Table of Contents.

iama

There are as many of these as specified in the lama record at offset 8.  May describe an Album/Collection.

OffsetLength InformationValue (example)
(0)4 Section signature iama
(4)4 Section length in bytes 68 00 00 00 (104)
(8)4 Associated Sections length DA 00 00 00 (218)
(12)4 How Many boma sections follow 02 00 00 00 (2)
(16)8 iama ID 9356DCFEC6CB1913
...4 ... ...

Associated Sections length counts from the beginning of this iama, and points to the next iama or Boundary (hsma) section.

Back to Table of Contents.

lAma Master for iAma records

Appears only once.  Contains a count of total iAma sections that follow. May be a list of Artists/Bands.

OffsetLength InformationValue (example)
(0)4 Section signature lAma
(4)4 Section length in bytes 64 00 00 00 (100)
(8)4 How Many iAma sections follow 03 00 00 00 (3)
...4 Zeroes 00 00 00 00 (0)

Back to Table of Contents.

iAma Items

There are as many of these as specified in the lAma record at offset 8.  Seems to be just Artist Names/Artwork links, in the associated boma structures.

OffsetLength InformationValue (example)
(0)4 Section signature iama
(4)4 Section length in bytes 78 00 00 00 (120)
(8)4 Associated Sections length 81 01 00 00 (385)
(12)4 How Many boma sections follow 02 00 00 00 (2)
(16)8 iAma Reference ID 984A02E3176E8762
...4 Unknown ...

Back to Table of Contents.

ltma Tracks Items Master

Appears only once.  Contains a count of total tracks in library (or how many itma sections to look for).

OffsetLength InformationValue (example)
(0)4 Section signature ltma
(4)4 Section length in bytes 5C 00 00 00 (92)
(8)4 How Many itma sections follow 03 00 00 00 (3)
...4 Zeroes 00 00 00 00 (0)

Back to Table of Contents.

itma Track Item Header

Each itma section is the start of the description of a track. 

OffsetLength InformationValue (example)
(0)4 Section signature itma
(4)4 Section length in bytes 78 01 00 00 (376)
(8)4 Unknown F3 0C 00 00 (3315)
(12)4 How Many (including this one? +) boma sections 12 00 00 00 (18)
(16)8 Track Persistent ID E260ED69DDF89F70
...4 Unknown ...
(42)2 Unchecked 00 00 (0 = is checked)
...4 Unknown ...
(62)2 Love (2), Dislike (3) or None (0) 02 00 (2 = Love)
...4 Unknown ...
(65)1 Stars (0 to 100/incr of 20) 14 (20) *One star
...4 Unknown ...
(86)2 Movements in Work (Classical) 00 00 00 00
(88)2 Movement of Work (Classical) 00 00 00 00
...4 Unknown ...
(168)4 Track Year CD 07 00 00 (1997)
(172)8 iama Reference ID 9356DCFEC6CB1913
(180)8 iAma Reference ID 984A02E3176E8762
...4 Unknown ...
(272)8 Track Persistent ID again (see note) 0000000000000000
...4 Unknown ...

NOTES:
The 8 byte hex-presented IDs needs to be byte-swapped, like all other little-endian numbers.
An 8 byte ID that seems to be linked to iama Sections at offset 172
An 8 byte ID that seems to be linked to iAma Sections at offset 180
Track Persistent ID again:  The track persistent ID was only repeated on the first itma record, and was all zeroes in subsequent itma records.
There is a lot more information here that I haven't figured out. 

Back to Table of Contents.

lPma

Playlists Master or lPma shows up only once.  This lists the number of playlists (lpma) sections.

OffsetLength InformationValue (example)
(0)4 Section signature lPma
(4)4 Section length in bytes 5C 00 00 00 (92)
(8)4 How Many lpma (Playlist Container) sections follow 08 00 00 00 (8)
...4 Zeroes 00 00 00 00 (0)

Back to Table of Contents.

lpma

Playlist Container or lpma exists for each Playlist.  This holds the number of associated (boma) sections for this playlist.

Observed as being followed by boma SubType 0x00C8 (Playlist Name), boma SubType 0x00CA (Unknown), optional boma SubType 0x00C9 (Smart Playlist Criteria), and a track count number of boma ipfa SubType 0x00CE (Playlist Track).

OffsetLength InformationValue (example)
(0)4 Section signature lpma
(4)4 Next Section start 44 3E 00 00 (324)
(8)4 Associated Sections Length 82 02 00 00 (642)
(12)4 How many boma sections follow 04 00 00 00 (4)
(16)4 How many tracks in playlist 03 00 00 00 (3)
...4 ... ...
(22)4 Playlist create date 1B EC 9B D9 (3650874395) *2019-09-09T11:46:35Z
...4 ... ...
(39)8 Playlist Persistent ID DCF3757CD5792CD7
...4 ... ...
(138)4 Playlist modified date 70 87 0C DA (3658254192) *2019-12-03T21:43:12Z
...4 ... ...

NOTE: There appears to be another date at offset (182) in some records, otherwise all zeroes, but I have not been able to figure out what it represents. 

Back to Table of Contents.

boma Data Container (common)

Every boma has a SubType, and what is inside each boma can be VERY different depending on that SubType.  The section length of a boma is indicated at a different offset from other section types.

Contrasting from the iTunes format:
It is useful to know that mhoh SubType numbers have the same purposes as the Apple Music boma SubTypes, though the offsets are slightly different. That said, iTunes has many more SubTypes, as that program could store much more than just Music files.

OffsetLength InformationValue (example)
(0)4 Section signature boma
(4)4 Depends, Always 0x14 14 00 00 00 (20)
(8)4 Section length in bytes A0 00 (160)
(12)4 BOMA SubType F6 01 (0x01F6)
...- ... ...

Back to Table of Contents.

boma SubType 0x0001 Track Numerics

OffsetLength InformationValue (example)
(0)4 Section signature boma
(4)4 Unknown 14 00 00 00 (20)
(8)4 Section length in bytes A0 00 (160)
(12)4 BOMA SubType 01 00 (0x0001)
...- ... ...
(108)4 Bit Rate 38 00 00 00 (56) *"56kb/s"
(112)4 Date Added 05 E0 DF D9 (3655327749) *2019-10-31 00:49:09
...- ... ...
(148)4 Date Modified 0B 63 FC D9 (3657196299)
...- ... ...
(176)4 Song Time (times 1000) D0 14 00 00 (5328) *5.328 seconds
...- ... ...
(316)4 File Size F9 A0 00 00 (41209)
...- ... ...

*Date is translated from the numeric value based on Number of seconds since 1 January 1904, Midnight UTC. 

Back to Table of Contents.

boma Widechar SubTypes

OffsetLength InformationValue (example)
(0)4 Section signature boma
(4)4 Unknown 14 00 00 00 (20)
(8)4 Section length in bytes A0 00 (160)
(12)4 BOMA SubType F8 01 (0x01F8)
...4 ... ...
(24)4 Bytelength of string 66 00 00 00 (102)
(36)4 String file:///Users/gary/Music/iTunes/iTunes%20Media/

Remember that the payload is in wide characters (UTF-16).  The resulting string length (in native) maybe as little as half.  In contrast with the iTunes itl format where there is an indicator of whether or not a payload is wide character or not. 

Back to Table of Contents.

boma Character SubTypes

Technically this is UTF-8, but it can be safely copied bytewise.

OffsetLength InformationValue (example)
(0)4 Section signature boma
(4)4 Unknown 14 00 00 00 (20)
(8)4 Section length in bytes A0 00 (160)
(12)4 BOMA SubType 38 00 (0x0038)
...4 ... ...
(24)4 Bytelength, pl = Section length subtracted from this length 66 00 00 00 (102)
pl (58)4 String <?xml version=...

Back to Table of Contents.

boma book SubTypes

boma book sections contain numbers and strings.  Strings are marked by 0x0101.  The four bytes prior to the 0x0101 will indicate the length of the string.  The byte following the 0x0101 will start the first character of that string.  These strings include the separate path components of the item.  The next string is 'file:///' The next string is 'Macintosh HD', or whatever the name of the install disk is.  The next string is the UUID of the install disk filesystem.  The final string is the path separator character, '/'.

OffsetLength InformationValue (example)
(0)4 Section signature boma
(4)4 book Start 14 00 00 00 (20)
(8)4 Section length in bytes A0 00 (160)
(12)4 BOMA SubType FC 01 (0x01FC)
...4 ... ...
(20)4 book signature 62 6F 6F 6B |book|
...4 ... ...
(72)4 String Length 05 00 00 00 (5)
(76)4 String Indicator 01 01 00 00 (0x00000101)
(80)4 String (5 long) Users

After reading each string, it is important to re-align to a 4 byte offset.  For the string shown above, 5 bytes were read.  That means 3 bytes must be discarded before the next four bytes can be read.

boma book sections are very long, and most paths are repeated in another boma section (not separated). 

There is one string indicated by a 0x0901, with the same setup as the 0x0101 string, except it is further preceeded by a 0x01F5. 

...4 ... ...
(364)4 String Indicator F5 01 00 00 (501)
(368)4 String Length 08 00 00 00 (8)
(372)4 String Indicator 01 09 00 00 (2305)
(376)4 String (8 long) file:///
...4 ... ...

There is one string indicated by a 0x0201, with the same setup as the 0x0101 string, except it is further preceeded by a 0x0501. 

...4 ... ...
(528)4 String Indicator 01 05 00 00 (1281)
(532)4 String Length DB 00 00 00 (219)
(536)4 String Indicator 01 02 00 00 (513)
(540)4 String (219 long) 20fec00...(not showing all of this)
...4 ... ...

Back to Table of Contents.

boma_ipfa

This block holds a single Track of a playlist.  It also has an identifier that is sometimes also found in Application.musicdb, which I am referring to (for now) as the ipfa ID. 

(0)4 Section signature boma
(4)4 Sub-Data Start 14 00 00 00 (20)
(8)4 Section length in bytes 58 00 00 00 (88)
(12)4 BOMA SubType CE 00 00 00 (0x00CE)
...4 ... 00 00 00 00
(20)4 ipfa SubSection Start ipfa
(24)4 Section length in bytes 44 00 00 00 (68)
(28)4 Short number varies 6D 00 00 00 (109)
(32)8 ipfa ID 002438F19F1A9224
(40)8 Track ID E260ED69DDF89F70
...4 ... 00 00 00 00
(64)8 ipfa ID 2 (see note) 002438F19F1A9224
...4 ... 00 00 00 00

Offset 64, ipfa ID 2, is usually zeroes, but when the ipfa is repeated here, it seems to indicate that this is the last Track ID within the playlist.

Back to Table of Contents.

boma Unknown

The preceeding are boma subtypes that follow the basic pattern at offsets (0), (8) and (12), but I have been unable to discern what data they hold.  Note, they also have differing lengths, so are probably not related.

Back to Table of Contents.

ssma

So far, I have only encountered this in Application.musicdb.  Marks (and pads) the end of the file. 

OffsetLength InformationValue (example)
(0)4 Section signature ssma
(4)4 Bytes to File End 60 00 00 00 (96)
...1 Padding 00 (0)

Back to Table of Contents.

Want To Help?

I am doing this work on a MacBook Air without enough space to put a large music library on here.  If you, the reader, are a user of macOS Catalina and happen to use Apple Music to store your music collection, I would love to run my assumptions above on your Library.musicdb file.

The file I am seeking is Music Library.musiclibrary which should automatically attach to an e-mail program as a .zip file. 

I would appreciate any musicdb files I can get.  Please send submissions to musicdb@vollink.com

Privacy:  The file(s) I'm asking for does not include any music.  It does include song file names, and how the user has rated and liked/disliked those songs, and all playlists saved.  The user's e-mail address may be embedded (I haven't found it yet, but it did exist in the older iTunes/itl format file).  Because the filenames include full paths, I will see the username of the Library (as in /Users/username/Music/iTunes/... ) I will NOT publish or share any files or information from any submission, unless I am explicitly given such permission.  I will use this to test the offsets and correllations I've found.  Unless explicitly asked to share, I will not retain any submitted file longer than 6 months (probably much less). 

This is a low traffic web site, and I will reply to any inquiry unless you specifically ask me not to reply.

Up to Table of Contents.

Acknowledgements

I would not have been able to get as far as I have without the fine work of Joseph Walton who produced titl.  That iTunes itl project in Java was key to my getting this file format cracked.

I also would not have been able to get as far as I have without the fine work of Jean Thomas who produced libitlp for reading iTunes itl format in C.  Despite all the section headers having different names, the formats (and tricks for hiding information) are remarkably similar.  Jean's faithful approach to parsing the actual format (and not an interpretation) helped immensely on what things to look for and what is probably safe to ignore.

Back to Table of Contents.

 

Valid CSS!