ITunesDB

From wikiPodLinux

The title of this article should be iTunesDB. The initial letter is capitalized due to technical limitations.

This page details the format of the binary files used on the iPod to keep track of the music it contains as well as its play history. Collectively we refer to these files as the iTunesDB however there are in fact a number of files, each with their own format, that make up this database.

Note that the information on this page is undergoing constant revision as new information comes to light. New versions of the iTunes or iPod software may cause drastic changes to the information in this document.

Basic information

The database directory on the iPod is /iPod_Control/iTunes/. This is a hidden folder (attrib +h), but is not otherwise protected.

Endian Note

With some exceptions, most of the data descriptions in this document are actually stored in the file in little endian byte order. Meaning that you have these representations of the data when looking at the file in a hex editor:

"01 00 00 00" = decimal 1
"10 00 00 00" = decimal 16
"00 01 00 00" = decimal 256

This means when you read the file, you need to reverse the bytes in memory to make sense of them. Keep this in mind when reading this document and trying to make sense of an iTunesDB using a hex editor.

The exceptions are noted where appropriate (hopefully). But if you are looking at a piece of a file and can't understand the value it holds, try reversing the byte order.

Note that this means the values themselves are in reversed byte order. The order of the fields described is the same as in this document. So if you see that field A is before field B in one of these tables, then when you look at the file with a hex editor, field A really will come before field B. This has confused some people who always reverse 4 byte chunks of data. If you're writing code to read/write these files, I recommend against *always* reversing the data in 4 byte blocks. This only makes the code more confusing in the long run.

iTunesDB file

The iTunesDB file consists of a sort of tree structure arranged into a flat file. Each "object" contains a header followed by some other objects. The header contains a couple of size fields that are used to define where the header ends and other objects begin. Here's the basic structure of it:

<mhbd> - This is a database
  <mhsd> - This is a list holder, which holds either a mhla
    <mhla> - This holds a list of albums
      <mhia> - This describes a particular Album Item
        <mhod> - These hold strings associated with an album title
        <mhod> - These hold strings associated with an artist name/title
      <mhia> - This is another album. And so on.
        <mhod>...
        <mhod>...
      ...
  <mhsd> - This is a list holder, which holds either a mhlt or an mhlp
    <mhlt> - This holds a list of all the songs on the iPod
      <mhit> - This describes a particular song
        <mhod>... - These hold strings associated with a song
        <mhod>... - Things like Artist, Song Title, Album, etc.
      <mhit> - This is another song. And so on.
        <mhod>...
        <mhod>...
      ...
  <mhsd> - Here's the list holder again.. This time, it's holding an mhlp
    <mhlp> - This holds a bunch of playlists. In fact, all the playlists.
      <mhyp> - This is a playlist.
        <mhod>... - These mhods hold info about the playlists like the name of the list.
        <mhip>... - This mhip holds a reference to a particular song on the iPod.
        ...
      <mhyp> - This is another playlist. And so on.
        <mhod>... - Note that the mhods also hold other things for smart playlists
        <mhip>...
        ...
      ...

And so on. What follows is a detailed description of these various headers.

Chunk Encoding

Each chunk of the file is encoded in the follow form:

The Chunk Type Identifier is intuitive: it's a 4-byte string letting you know what you're working with.

The End of Type-specific Header tells you where, relative to offset 0, the header section for this chunk ends. The header starts at offset 12 and runs through to the end of the type-specific header.

The End of Chunk or Number of Children is slightly confusing. Usually, it's an "End of Chunk" offset: what the last offset in the current chunk is. That is, it tells you how long this chunk and all its children are. There are two big exceptions to this: "mhlt" and "mhlp" chunks. In both of these, this number is how many top-level children the mhl[pt] chunk has.

Database Object

The Database Object has two or three children, which are Data Sets.

DataSet

1 = Track list - contains an MHLT
2 = Playlist List - contains an MHLP
3 = Podcast List - optional, probably. Contains an MHLP also.  
This MHLP is basically the same as the full playlist section, 
except it contains the podcasts in a slightly different way. 
See the Playlists section.
4 = Album List, first seen with iTunes 7.1.
5 = New Playlist List with Smart Playlists, first seen with iTunes 7.3.

Depending on the type of Data Set, it will contain either a Track List child or a Playlist List child. Order is not guaranteed. Example files have contained the type 3 MHSD before the type 2 MHSD. In order for the iPod to list podcasts the type 3 Data Set MUST come between the type 1 and type 2 Data Sets.

TrackList

The Track List has Track Items as its children. The number of Track Items is the same as the number of songs.

Track Item

The MHIT is followed by several Data Objects which have string types. At minimum, it must have a Location type MHOD, in order to tell the iPod where the file is located on the iPod itself. It always has a FileType MHOD as well, although it's not totally necessary.

Playlist List

The Playlist List has all the playlists as its children. The very first playlist *must* be the Library playlist. This is a normal playlist, but it has the special "hidden" bit set in it, and it contains all the songs on the iPod (in no particular order).

Playlist

The structure of the Playlists are different than most others. Each Playlist looks like this, conceptually:

   <mhyp>
      <mhod type=1>Playlist Name</mhod>
      <mhod type=50>Smart Playlist Info</mhod> (optional)
      <mhod type=51>Smart Playlist Rules</mhod> (optional)
      ...
      <mhip>Playlist Item</mhip>
             <mhod type=100>Position Indicator</mhod>
      <mhip>Playlist Item</mhip>
             <mhod type=100>Position Indicator</mhod>
      ...
   </mhyp>

The point being that these are all considered in the MHYP for the size calculations. However, in the "Data Object" child count, ONLY those MHODs that come before the first MHIP are counted. The "position indicators" are children of the MHIPs (older firmwares had a bug in this respect).

List Sort Order

1 - playlist order (manual sort order)
2 - ???
3 - songtitle
4 - album
5 - artist
6 - bitrate
7 - genre
8 - kind
9 - date modified
10 - track number
11 - size
12 - time
13 - year
14 - sample rate
15 - comment
16 - date added
17 - equalizer
18 - composer
19 - ???
20 - play count
21 - last played
22 - disc number
23 - my rating
24 - release date (I guess, it's the value for the "Podcasts" list)
25 - BPM
26 - grouping
27 - category
28 - description
29 - show
30 - season
31 - episode number

Podcasts

The podcasts playlist is organized slightly differently, in the Type 3 MHSD. For example:

mhyp (MHODs: 2, MHIPs: 5, hidden: 0, list sort order: 0x18)
  mhod (type: 1, string: 'Podcasts')
  mhod (type: 100)
  mhip (MHODs: 1, groupflag: 256, groupid: 8232, trackid: 0, timestamp: 0, groupref: 0)
    mhod (type: 1, string: 'Example podcast')
  mhip (MHODs: 1, groupflag: 0, groupid: 8233, trackid: 8230, timestamp: 3206828281, groupref: 8232)
    mhod (type: 100)
  mhip (MHODs: 1, groupflag: 0, groupid: 8234, trackid: 8226, timestamp: 3206828379, groupref: 8232)
    mhod (type: 100)
  mhip (MHODs: 1, groupflag: 0, groupid: 8235, trackid: 8228, timestamp: 3206828327, groupref: 8232)
    mhod (type: 100)
  mhip (MHODs: 1, groupflag: 0, groupid: 8236, trackid: 8224, timestamp: 3206828394, groupref: 8232)
    mhod (type: 100)

The first mhip (probably identified by groupflag==256) contains the name of a podcast which will then appear as a submenu of the Podcasts menu on the iPod. The other mhips (which reference the episodes in that podcast) seem to use the groupref field to link to their 'parent' mhip (using the groupid field). At the same time the groupids of the episodes are unique as well, but don't seem to be used anywhere else.

Playlist Item

For purposes of size calculations, Playlist Items have no children. However, every Playlist Item is invariably followed by a Data Object of type 100, which contains nothing but a number that is used to order/sort the playlist. See the Playlist description above for more information.

Please note that starting with iTunes 4.9 (mhbd file version number 0x0d) the Type 100 MHOD following the Playlist Item is considered a child of the Playlist Item and is included into the size calculation. The old behaviour was probably a bug in iTunes.

Data Object

The Data Object is a complex beast. It's used in many places in the iTunesDB file, and there are many forms of it. We'll cover the forms one by one.

String Types

The simplest form of MHOD. These are any MHOD with a "type" that is less than 15.

Podcast URLs (types 15-16)

Introduced in db version 0x0d, MHOD's with type 15 and 16 hold the Enclosure and RSS URL for the Podcast. The string is probably UTF-8, but only Unicode symbols U+0000 through U+007F (a.k.a ASCII) have been observed.

Chapter Data (type 17)

The chapter data is a style of data that's very, very out of place in this file, but is here nonetheless. It defines where the chapter stops are in the track, as well as what info should be displayed for each section of the track. It seems that the format of this mhod changed significantly over time. The following is analysed from an iTunesDB version 0x13 (iTunes 7.0).

Chapter Data mhod details

Chapter Data Atom Layout

The atoms appear to be arranged in the following layout:

sean (exactly one atom of this type is present,
        childcount is number of chapters + 1 for the hedr atom)
   chap (chapter indicator, childcount is always 1)
      name (contains a UTF16 string with name of the chapter or Artist name, 
            childcount is always 0)
   chap (next chapter)
      name (and so on)...
   ...
   hedr (signals the end, childcount is always 0)

There are multiple "chap" entries, one for each chapter.

Older DBs seem to have a different, more complex layout, like this:

sean (?)
 chap (chapter indicator ?)
  name (contains a UTF16 string with name of the chapter or Artist name)
  ploc (position/location?)
   trak (specifies track number, perhaps? This should be obvious after looking at some examples)
  urlt (contains a UTF16 string with the name of the song in the chapter or some kind of subname)
  url  (contains a UTF8 string with a URL)
 chap (next chapter)
  name (and so on)...
 ...
hedr (signals the end)

Chapter Data Atoms

Each atom consists of the following:

Caution: This is different to all other structures in the file, where the
         name always comes first. Here, the length is the first word of the atom!

Chapter Data String Atoms (UTF16)

UTF16 String entries in these atoms (like 'name') fit the following mold:

Chapter Data String Atoms (UTF8)

UTF8 entries fit the following mold:

Chapter Data hedr Atom

Smart Playlist Data (type 50)

A slightly more complex MHOD. These are any MHOD with a "type" that is 50.

This MHOD defines the stuff in the Smart playlist that is not the "rules". Basically all the checkboxes and such. It's pretty straightforward.

When the Reverse Limit Sort flag is set, the sort will be reversed. So most recently added becomes least recently added, and highest rating becomes lowest rating, and so on. It's just reversing the sorted list before applying the limit to it.

Smart Playlist Rules (type 51)

The most complex (and annoying) MHOD. These are any MHOD with a "type" that is 51.

This MHOD defines all the rules in the Smart Playlist. It is not straightforward at all. In fact, I have to break it down into subtypes itself.

If you've used iTunes, you know what a rule looks like. "Rating is less than 3 stars", for example. The rule consists of three parts.

The first part is the "field". In our example, the field is "Rating". It's the very first thing in the rule and the first pull down box in iTunes. Simple enough.

The second part is the "action". In our example, the action is "is less than". In iTunes, it's the second pull down box. Also pretty simple.

The final part is the "value". In our example, it'd be "3 stars". Since stars are always multiplied by 20 for whatever reason, it'd be "60" for our example, in the iTunesDB file.

We'll cover these one at a time.

Important Note about endian-ness

Smart Playlist Rules MHODs are NOT wholly little-endian. Everything after the "SLst" to the end of the MHOD is big-endian. This is important to remember, especially when dealing with the Action types.

Smart Playlist Rule Fields

Fields are easy. Much more so than Actions. The "expected comparison" column will make more sense when you see the Values explained below.

Special Note about Grouping and Playlist fields - They don't work with Live Updating on 3rd gen iPods, yet. This might get fixed in a future firmware release. Maybe.

Smart Playlist Rule Actions

The Action type is a 4 byte field. It is a bitmapped value, meaning that each bit of these four bytes has a different meaning.

This is complicated, obviously, and iTunes abstracts this all by defining various actions for various fields. Here's the complete possible list of actions, including those that iTunes does not have but which still work on the iPod:

Smart Playlist Rule Values

Values are generally pretty straightforward. For String rules, it's a string. For Integer and Timestamp rules, it's a bit more complicated. Furthermore, the "In the Last" type action requires a Units value as well.

So there are two major rule formats, the String Rule and the Non-String Rule.

For integer type rules, the from and to values are the ones that you care about, the date = 0, and the units = 1.

Example: BPM is less than 150
field = 0x23 (BPM)
action = 0x00000040 (is less than)
from and to value = 150
from and to date = 0
from and to units = 1

Example: BPM is in the range 70 to 150
field = 0x23 (BPM)
action = 0x00000100 (is in the range)
from value = 70
to value = 150
from and to date = 0
from and to units = 1

For binary and type rules, the from and to values are the ones that you care about, the date = 0, and the units = 1.

Example: Video Kind is TV-Show
field = 0x3c (Video Kind)
action = 0x00000400 (Is / Is Not)
from and to value = 0x0040
from and to date = 0
from and to units = 1

Example: Video Kind is not TV-Show
field = 0x3c (Video Kind / mediatype)
action = 0x00000400 (Is / Is Not)
from and to value = 0x0e22
from and to date = 0
from and to units = 1

For the latter one would expect a value of 0x0022 (either Movie or Music Video). The additional 0x0e00 in the mask hints for further video types to come.

Timestamp type rules use the same format as integer type rules, only the from and to values are Mac timestamps. This is the number of seconds since 1/1/1904.

Example: Date Added is in the range 6/19/2004 to 6/20/2004
field = 0x10 (Date Added)
action = 0x00000100 (is in the range)
from value = bcfa83ff (6/19/2004) 
to value = 0xbcfbd57f (6/20/2004)
from and to date = 0
from and to units = 1

For "in the last" type rules, the from and to values are set to a constant of 0x2dae2dae2dae2dae. The from dates become the value, and the from units becomes the unit you're measuring in. The way to think of this is that it's saying "Today (2dae) plus this number of units".

Example: Last Played Time is in the last 2 weeks.
field = 0x17 (Last Played Time)
action = 0x00000200 (is in the last)
from value = 0x2dae2dae2dae2dae
from date = -2
from units = 604800 (number of seconds in a week)
to value = 0x2dae2dae2dae2dae
to date = 0
to units = 1

That rule is saying "Today minus 2 times this number of seconds" which is 2 weeks before "now", whatever now happens to be.

If you're creating your own rules in an iTunesDB, you may find it more convienent to leave the units set to "1" all the time and just put the number of seconds into the date field. This is perfectly acceptable and the iPod can handle it just fine. If you're not sure how to make a particular rule, create it or a similar one in iTunes and put it on the iPod, then examine the iTunesDB file to see how it did it.

But if you're programming the iPod side of things, you need to be able to correctly understand the units field and deal with it accordingly. The best way to do this is to always compare the contents of the field in question with (value+date*unit), and replacing the "value" with the current timestamp ("now") when it is equal to 3,291,618,603,768,360,366 (0x2dae2dae2dae2dae). This will work for *all* integer and timestamp comparisons if done correctly. It's also exactly what the Apple iPod firmware does. It's also why you have to set the time/date on an iPod for these smart playlists to work correctly (3rd gen and up).

Also, remember that all timestamps should be dealt with in Apple format, which is the number of seconds since 1/1/1904.

Putting It All Together

So, now you know how to create a given rule. All that's left is to put all the rules together and shove them into a type 51 MHOD.

Library Playlist Index (type 52)

The type 52 MHOD is only found as a child of the Main Library Playlist. It is an index of the mhit's ordered by the major categories in the Browse menu. The purpose of these mhod's is to speed up the operation of the Browse menu itself. This is how it displays the information so quickly when selecting one of the major categories, it's all presorted for the iPod in these MHOD's.

Note that this MHOD is not mandatory, however the iPod menu system will operate much slower without it (on large libraries), as it will have to build the information provided here on the fly. Therefore it is recommended to build this MHOD for anything more than trivial numbers of songs.

Essentially, every MHIT is numbered from 0 to the total number of songs-1. The type 52 MHOD contains a list of these index numbers using one of the strings contained in these MHIT's, ordered alphabetically.

To build one of these, take all your songs, order them alphabetically by one of these fields, then simply insert the index numbers of the ordered songs into the type 52 mhod.

(Note that the above list roughly matches the limit sort list.. I think that these lists are actually the same in some way.)

Type 53

The type 53 MHOD is only found as a child of the Main Library Playlist.

I don't know how these are associated with the corresponding MHOD52s, I would guess by the sort order field, but I'm not sure because they always turn up immediately after their MHOD52. Could someone please have a look at this and the other yet unknown things above?

Type 100

There are a couple forms of Type 100 MHOD's (that we know of).

The first form is at the beginning of every playlist (before any MHIP entries). Only iTunes puts it there, and only iTunes uses it. It contains information on what columns to display, and what size to display them as, when displaying the playlist in iTunes when the iPod is in a manual mode. This is absolutely optional. The iPod itself doesn't appear to use it in any way.

The second form is after every MHIP in a playlist, there's a small type 100 MHOD. It gives an order number to the song immediately before it. It is not optional, but it's also not particularly useful.

Playlist Column Definitions Header

Column Definition

Each column definition only consists of an ID for the playlist and the sort direction for the column. The order they appear in this MHOD are the order they appear in iTunes, from left to right. The first two columns are always song position and title, in that order.

Playlist Order Entry

Album List

Album List, first seen with iTunes 7.1. It was seen on iPod nano 2nd gen and Shuffle 2nd gen restored with iTunes 7.1.1, but not on iPod Video 30GB(restored with the same version of iTunes).

The Album List has Album Items as its children. The number of Album Items is the same all albums on iPod.

Album Item

Usually mhia has two child strings: album title and artist name.

Play Counts file

The play count information, since the last sync with iTunes, is stored in the "Play Counts" file.

The play counts file is deleted by the iPod and rebuilt whenever it detects that the iTunesDB file has changed (immediately after a sync). So writing anything into this file yourself is more than a little pointless, as the iPod simply will overwrite it. This is the iPod's way of sending information back to iTunes (or whatever program you happen to sync with).

When iTunes syncs, the first thing it does is read the values that the iPod has put into this file, and merges them back into the iTunes Library.

Play Count Header

The play count header indicates a valid play count file and specifies how many entries follow and the size of each entry record. The is an entry record for each song on the iPod; the entry position corresponding to the position of the song in the iTunesDB.

Play Count Entry

The entry record contains the play count data for each song, one record exists for each song on the iPod.

Older firmware versions had set the last played time and the rating to "zero" in this database, unless you changed either one on the iPod itself. Thus you could ignore zero entries.

Newer firmwares, however, set last played time and rating to be the same as what is in the iTunesDB file, by default. Meaning that you can't ignore zero entries for rating anymore (somebody could set the rating to a zero rating on the iPod). So you must compare the rating in Play Counts and the rating in iTunesDB to determine if it has actually changed and you need to update it on the PC (in a program on the PC that talks to the iPod that is).

On The Go Playlist files

Because the OTG Playlist uses track indexes, it is only valid until the next time the iTunesDB changes. This means that you should read any OTGPlaylist files on a sync and create proper Playlists for them in the iTunesDB file or do something else with them. After you change the iTunesDB file, they won't be valid anymore.

iTunes handles this by deleting the files once they are read, during an autosync. Apple's iPod firmware also deletes these files if the iTunesDB changes and it detects it, after you undock the iPod. iTunes will read them and create real playlists from them before it syncs anything. Note that in any sync operation, this *must* occur before the iTunesDB changes, otherwise you'll lose the playlist.

Equalizer Presets file

The EQPresets file consists of a Presets Container to specify the number of EQPresets in the file, and then a series of EQ Presets as its children.

Equalizer Presets Container

Equalizer Preset

Each EQ Preset contains two actual presets. The first one is the representation you see in iTunes - a 10 band Equalizer. iTunes actually displays the given bands slightly wrong. The real values it uses are: 32Hz, 64Hz, 128Hz, 256Hz, 512Hz, 1024Hz, 2048Hz, 4096Hz, 8192Hz, 16384Hz

The EQ Preset also contains a 5 band version of the exact same preset. The algorithim it uses to determine this from the 10 band preset is not known, but you can assume it uses some form of curve fitting method. In any case, the 5 band version would be the EQ setting that the iPod actually would use, if it actually read and used this file as it obviously was supposed to do. The reason for this is that the iPod's audio DSP is capable of applying a 5 band EQ directly.

Artwork Database

The ArtworkDB file is only found on iPod Photos. It is created by iTunes 4.7 and up. This file, along with the thumbnail files iTunes creates, are what allows photos and album art to be displayed on the iPod Photo.

The Photo Database file is similar to the ArtworkDB file. The Photo Database stores photos you add manually, the ArtworkDB stores album artwork for displaying when playing music.

-Images are concatenated into *.ithmb files (thumbnails, basically).

-Large thumbnails are stored separately from small thumbnails.

-Large thumbnails are 140x140, small thumbnails are 56x56 - raw RGB565 packed color binary streams (bytes are swapped, little endian).

ArtworkDB Database layout:

<mhfd>
 <mhsd> (index = 1)
   <mhli>
     <mhii>
       <mhod> (type = 2) Info about full size thumbnail
         <mhni>
           <mhod> (type = 3)
       <mhod> (type = 2) Info about 'now playing' thumbnail
         <mhni>
           <mhod> (type = 3)
     <mhii>
       <mhod> (type = 2)
         <mhni>
           <mhod> (type = 3)
       <mhod> (type = 2)
         <mhni>
           <mhod> (type = 3)
      ...
 <mhsd> (index = 2)
   <mhla>
 <mhsd> (index = 3)
   <mhlf>
     <mhif>
     <mhif>
     ...

Photo Database

The Photo Database is created by iTunes, and is stored in "/Photos/Photo Database". The Photo Database looks similar to the ArtworkDB but has additional entries in the mhla object (mhba and mhia, see below) as well as different Thumbnails. The mhiis in the Photo Database look like this for example:

mhii (children: 5, id: 117, srcimgsize: 0)
  mhod (type: 5)
    mhni (children: 1, corrid: 1, ithmb offset: 0, imgsize: 1567395, imgdim: 0x0)
      mhod (type: 3, length: 80, string: u':Full Resolution:2005:08:06:simg3609.jpg')
  mhod (type: 2)
    mhni (children: 1, corrid: 1019, ithmb offset: 7603200, imgsize: 691200, imgdim: 0x2c801e0)
      mhod (type: 33554435, length: 42, string: u':Thumbs:F1019_1.ithmb')
  mhod (type: 2)
    mhni (children: 1, corrid: 1020, ithmb offset: 851840, imgsize: 77440, imgdim: 0xa500e1)
      mhod (type: 33554435, length: 42, string: u':Thumbs:F1020_1.ithmb')
  mhod (type: 2)
    mhni (children: 1, corrid: 1009, ithmb offset: 27720, imgsize: 2520, imgdim: 0x29001f)
      mhod (type: 33554435, length: 42, string: u':Thumbs:F1009_1.ithmb')
  mhod (type: 2)
    mhni (children: 1, corrid: 1015, ithmb offset: 251680, imgsize: 22880, imgdim: 0x7b0058)
      mhod (type: 33554435, length: 42, string: u':Thumbs:F1015_1.ithmb')

The type 5 mhod references the full resolution image and is probably only there if the corresponding check box in iTunes was checked. The type 2 mhods reference different types of thumbnail versions:

1. The first thumbnail (imgdim: 0x2c801e0, which decodes as 712x480) is of dimension 720x480 and in YUV 4:2:2 format, interlaced. It consists of two half-frames concatenated together. This is probably used for the TV-out (dimension, color format and interlacing look like NTSC).
2. The second thumbnail (imgdim: 0xa500e1, which decodes as 165x225) is of dimension 176x220 and in RGB 565 format, but rotated 90° CCW. This is the image that is actually displayed on the iPod screen.
3. The third thumbnail (imgdim: 0x29001f(41x31)) is 42x30, RGB 565 with swapped bytes (e.g. it's more like GBRG 3553). This is the image that is used as a thumbnail.
4. The fourth thumbnail (imgdim: 0x7b0058(123x88)) is 130x88, RGB 565 with swapped bytes

There is a *.ithmb file per resolution (in the directory "/Photos/Thumbs/"), that concatenates all thumbnails with that resolution.

On an iPod video (5G) there are 4 different thumbnails type:

1. 720x480 interlaced UYVY (YUV 4:2:2) - used for TV output - 691200 bytes each single thumbnail
2. 320x240 byte swapped RGB565 - used for fullscreen on the iPod - 153600 bytes each single thumbnail
3. 130x88 byte swapped RGB565 - used on the iPod during slideshow, when current photo is displayed on TV - 22880 bytes each single thumbnail
4. 50x41 byte swapped RGB565 - used on the iPod when listing and during slideshow - 4100 bytes each single thumbnail

Dimensions of the fields in the Photo Database are very important. Only one total length field or one padding field with wrong value could be enough to make the Photo Database file completely unusable: then nothing will be displayed on the iPod, no photo albums, no photos.

Here follows a complete structure for a Photo Database file working on an iPod video 5G:

 'mhfd', 132, 1384, 0, 1, 3, 0, 102, 0, 0, 0, 0, 2, 0, 0, 0, 0
 'mhsd', 96, 1276, 1
 'mhli', 92, 1
 'mhii', 152, 1088, 5, 100, 102, 0, 0, 0, 0, 3221487006, 3221487006, 0
 'mhod', 24, 216, 5, 0, 0
 'mhni', 76, 192, 1, 1, 0, 0, 0, 0, 0, 0
 'mhod', 24, 116, 3, 0, 2
 78, 2, 0, ':Full Resolution:2006:01:22:DSC00090.JPG'
 'mhod', 24, 180, 2, 0, 0
 'mhni', 76, 156, 1, 1019, 0, 691200, 0, 0, 480, 712
 'mhod', 24, 80, 3, 0, 2
 42, 2, 0, ':Thumbs:F1019_1.ithmb'
 'mhod', 24, 180, 2, 0, 0
 'mhni', 76, 156, 1, 1024, 0, 153600, 0, 0, 240, 320
 'mhod', 24, 80, 3, 0, 2
 42, 2, 0, ':Thumbs:F1024_1.ithmb'
 'mhod', 24, 180, 2, 0, 0
 'mhni', 76, 156, 1, 1015, 0, 22880, 0, 0, 88, 123
 'mhod', 24, 80, 3, 0, 2
 42, 2, 0, ':Thumbs:F1015_1.ithmb'
 'mhod', 24, 180, 2, 0, 0
 'mhni', 76, 156, 1, 1036, 0, 4100, 0, 0, 41, 53
 'mhod', 24, 80, 3, 0, 2
 42, 2, 0, ':Thumbs:F1036_1.ithmb'
 'mhsd', 96, 660, 2
 'mhla', 92, 2
 'mhba', 148, 232, 1, 1, 101, 0, 65536, 0, 0, 0, 0, 0, 0, 0, 100
 'mhod', 24, 44, 1, 0, 1
 7, 1, 0, 'Library'
 'mhia', 40, 40, 0, 100
 'mhba', 148, 240, 1, 1, 102, 0, 393216, 0, 0, 0, 0, 0, 0, 0, 101
 'mhod', 24, 52, 1, 0, 3
 13, 1, 0, 'A Photo Album'
 'mhia', 40, 40, 0, 100
 'mhsd', 96, 684, 3
 'mhlf', 92, 4
 'mhif', 124, 124, 0, 1019, 691200
 'mhif', 124, 124, 0, 1015, 22880
 'mhif', 124, 124, 0, 1024, 153600
 'mhif', 124, 124, 0, 1036, 4100

Data File Object

DataSet

This is basically the same as the MHSD element in the iTunes DB.

Depending on the index, the Data Set either contains an Image List (mhli) child, an Album List (mhla) child or a File List child (mhlf).

Image List

The Image List has Image Items as its children. The number of Image Items is the same as the number of images.

Image Item

MHOD type 2, Annoyingly, not a string like it is in iTunesDB... but still defines location of the file in question, sorta: its mhni child record contains everything that is needed about the image file.

header_size = 0x18
total_size
type = 2
unk1 = 0
unk2 = 0

Image Name

The correlation ID gives us the name of the file containing the image. For example, if the correlation ID is 1016 in decimal, then the corresponding filename will be F1016_1.ithmb.

In general, (vertical padding +image height) ~ pixmap height - usually within one or two pixels, probably due to rounding error. For instance, on a PhotoPod, an original image with dimensions 1200h x 1600v will have an NTSC image with image height=480, image width=558, vertical padding=0, and horizontal padding=162, with 558+162 = 720, the actual width of the pixel map. For an image scaled to be contained entirely within the pixel map, such as the video image or the full-screen image the padding values are basically the total width of the black bars.

For the smallest thumbnails, you can have negative values for padding, because the pixel map is scaled to be contained within the image - you get a central "slice" with no black bars.

As noted, there appear to be some rounding errors when the padding values are calculated, as the sums are sometimes off by 1 to 2.

There is no indication in this object what the pixel format, actual pixel map dimensions or rotation of images will be, so this must be entirely derived from the image size.

Here are the dimensions and formats for all known image sizes:

where:

• UYVY is a byte stream where U,Y0,V,Y1 creates two YUV pixels of Y0,U,V and Y1,U,V, interlaced, all even fields, then all odd fields.
• RGB565_LE is a stream of byte-swapped 16-bit pixels ordered from top->bottom, left->right
• RGB565_BE_90 is a stream of 16-bit pixels ordered right to left, top to bottom

The "full screen" images are rotated because the iPod displays used are actually portrait, not landscape, and this format is just a memory dump of the frame buffer memory.

Album List

The Album List has no children in the case of the ArtworkDB file, and 1 or more children for the Photo Database: 1 child for the Photo Library and possible some more children for additional photo albums.

For the Photo Database the layout looks like this, for example:

mhsd (type: 2)
  mhla (children: 3)
    mhba (number of mhods: 1, number of mhias: 5, unk3: 0x10000)
      mhod (string: "My Pictures")
      mhia (image id: 100)
      mhia (image id: 101)
      mhia (image id: 102)
      mhia (image id: 103)
      mhia (image id: 104)
    mhba (number of mhods: 1, number of mhias: 2, unk3: 0x60000)
      mhod (string: "Folder A")
      mhia (image id: 100)
      mhia (image id: 101)
    mhba (number of mhods: 1, number of mhias: 3, unk3: 0x60000)
      mhod (string: "Folder B")
      mhia (image id: 102)
      mhia (image id: 103)
      mhia (image id: 104)

In this case "My Pictures" was the folder that was selected in iTunes to synchronize photos with, and it contained 2 folders "Folder A" and "Folder B" with 2 and 3 photos respectively. The iPod will show this on its Photo menu as a submenu called "Photo Library" (containing all 5 photos), a submenu called "Folder A" with the first 2 photos and a submenu "Folder B" with the other 3 photos.

Note that the string MHODs are zero-padded to a length that is a multiple of 4 so that in the Photo Database all objects start on a 4-byte boundary. The iPod won't list any photo or photo album otherwise.

Photo Album

An MHBA allows you to setup an Album of photos (grouping photos together in an album). The MHBA is implemented similar to playlist.

These are only supported on models that support photos.

The MHBA has several children: The first is a MHOD containing the album name (which is ignored if the MHBA is the Photo Library). After that there are several MHIAs that define which image to show in that album. The MHBA that is marked as Photo Library contains one MHIA record for every photo on the iPod.

On the new video iPods and nanos of recent vintage, the MHBA has a second MHOD as a child which contains a string of which specifies the transition effect configured in iPhoto for the slideshow associated with this album. Apparently, the iPod ignores the slideshow settings that come from iPhoto, except for the slideshow soundtrack.

Album Item

File List

The File List has Files (Images) as its children.

File (Image)

Data Object

The MHODs found in the ArtworkDB and Photo Database files are significantly different than those found in the normal iTunesDB files.

There are 2 groups of types of MHODs in the ArtworkDB: container MHODs contain a MHNI as a child, while 'normal' string MHODs contain a string.

Attention: Sometimes seems that the MHBAs in the new video and nano iPods' Photo Database have a second MHOD child which, although being identified by a type of 2, is a string (and not container) MHOD. This second string MHOD in photo album is usually found in Photo Database files generated on Macs, probably by iPhoto, and contains an UTF-8 string describing a transition effect such as "Dissolve". However in Photo Database files generated on PCs for example by iTunes 6 for an iPod video 30Gb this does not happen, and there is only one type-1 string MHOD as child, just like with iPod Photo/Color Photo Database files.

Container MHODs

MHODs with type 2 contain a MHNI that (contains a type 3 MHOD that) references a thumbnail. MHODs with type 5 contain a MHNI that (contains a type 3 MHOD that) references a full resolution image (in the Photo Database).

String MHODs

The content of string MHODs (probably all types except 2 and 5, although only 1 and 3 have been observed so far) is structured again with something like a sub-header:

iTunesSD file

The iTunesSD file is in a big-endian byte order. It consists of a header followed by a bunch of entries, one after the other. The format is much simpler than the iTunesDB. Only the iPod Shuffle is known to use this file at the moment. The Shuffle uses only this file for playing songs, but nevertheless a valid iTunesDB must be present on the device. When connecting to iTunes, only the iTunesDB is read back, not the iTunesSD.

iTunesStats File

This seems to be the equivalent of the "Play Counts" file, where the Shuffle stores data that iTunes reads back in. Strangely, this file *is* byte reversed (little-endian). So a decimal 18 would be represented as 0x12 0x00 0x00, for example.

iTunesShuffle File

This file contains a list of the track index numbers, one after another. Each number is 3 bytes in length, and in little endian byte order (reverse byte order).

The reason for this file is that it ensures that although the tracks are shuffled, they remain in the same order until you resync the iPodShuffle. That way you can turn the device on and off as many times as you like, and although the songs are shuffled, you can still back up and so forth.

iTunesShuffle can either be created by the application, or the iPod Shuffle will automatically create it when necessary. It can also be recreated on the Shuffle by pressing the play/pause button three times within one second, thus reshuffling the order of songs on the fly.

It also appears that the songs are automatically reshuffled if, while in shuffle playback mode, the player reaches the end of the playlist. So even if an application creates the iTunesShuffle file, the iPod Shuffle can and will replace it.

iTunesPState File

This file keeps track of the current playback state of the iPod shuffle. It contains seven 24-bit little endian values (i.e. 3 bytes per number, reverse byte order). The iPod re-creates the file with default if it is missing.

iTunesControl File

It is quite big (several MB), and has no obvious header identifier.
It seems to have a single data section written over and over again which equals 256 bytes each.
You can see a pattern in the hex data section of growing word values.

iTunesPrefs File

It seems to have a fixed length of 236 bytes.

winPrefs File

It seems to have a fixed length of 16 bytes.

DeviceInfo File

This file has info on the iPod user's computer.
It seems to have a fixed length of 1536 bytes.
Each field is zero padded and in UTF-16LE text format.

iEKInfo File

New file I found recently on a 3G iPod. Unknown purpose. The file looks to be big endian, and similar to the chapter info records found inside podcasts with chapters (as well as similar to the chapter info structures found in the iTunesDB file. One odd characteristic is that the file appears to contain no 4 character identifier at the beginning. Similar in structure to standard MPEG 4 metadata layouts.

some more information on the structure of this file (http://www.adg.us/article188.html)

Info copied from above webpage (adg.us):

FILE HEADER:
============
 <uint32> - Size of file data (excluding this 64-byte header)
 <60-zero-bytes> - Padding?
FILE DATA BEGINS HERE:
======================
 <12-zero-bytes> - Padding?
 <ATOM:"sean">
   <uint32> - ID/number (should be 1)
   <uint32> - No. of subatoms (should be at least 3)
   <uint32> - Four zero bytes
   <ATOM:"sym ">
     <uint32> - ID/number (should be 1)
     <uint32> - No. of subatoms ("sess" subatoms) - I expect it should be
                at least 1
     <uint32> - Four zero bytes
     ONE OR MORE "sess" ATOMS:
     =========================
       <ATOM:"sess">
         <uint32> - Session ID - Used to identify which keys are encrypted
                    with this session
         <uint32> - No. of subatoms - Should be 2 (one "valu" subatom, one
                    "index" subatom)
         <uint32> - Four zero bytes
         <ATOM:"valu">
           <uint32> - ID/number (should be 1)
           <uint32> - No. of subatoms (should be zero)
           <uint32> - Four zero bytes
           <128 bytes of binary data> - The session key or something???
           <128 zero bytes> - I don't know if all sessions are structured
                              this way or not...  These last 128 bytes
                              are ignored for playback purposes.
         --- ("valu")
         <ATOM:"indx">
           <uint32> - ID/number (should be 1)
           <uint32> - No. of subatoms (should be zero)
           <uint32> - Four zero bytes
           <uint32> - Every index I've seen thus far has the value six
                      ('6') here.  Changing this broke playback on my
                      iPod, so it's important for playback.
         --- ("indx")
       --- ("sess")
     ===
   --- ("sym ")
   <ATOM:"user">
     <uint32> - User ID (a.k.a. DSID)
     <uint32> - Number of keys, or no. of "key " subatoms
     <uint32> - Four zero bytes
     ONE OR MORE "key " ATOMS:
     =========================
       <ATOM:"key ">
         <uint32> - Key ID or key number
         <uint32> - Number of subatoms (should be 1)
         <uint32> - Four zero bytes
         <ATOM:"valu">
           <uint32> - Session ID used to encrypt this key's contents
           <uint32> - No. of subatoms (Should be zero)
           <uint32> - Four zero bytes
           <16 bytes of binary data> - The key data, encrypted with the
                                       specified session?
         --- ("valu")
       --- ("key")
     ===
   --- ("user")
   THREE "guid" ATOMS: (Are there ever more?)
   ===================
     <ATOM:"guid">
       <uint32> - GUID ID or number (should be one)
       <uint32> - Number of subatoms (should be zero)
       <uint32> - Four zero bytes
       <GUID Data Bytes> - 256 bytes for the first 'guid' atom...
         The first 9 bytes, if changed, appear to break playback on my
         iPod.  The next three bytes contained data, but I could change
         it without breaking playback.  Byte number 12 (position 11 if
         the first byte is at position zero) was 1 for a while on my
         iPod, but later was 2.  I didn't check to see if this changed
         when I moved the iPod from one host computer to another.
         On a 4G iPod, the first 8 bytes of this field are the iPod's FireWire ID.
     --- ("guid")
     <ATOM:"guid">
       <uint32> - GUID ID or number (should be two)
       <uint32> - Number of subatoms (should be zero)
       <uint32> - Four zero bytes
       <GUID Data Bytes> - 256 bytes for the second 'guid' atom...
         This has got to be a global unique identifier based on which
         host computer installation of iTunes the iPod is associated
         with.  It changed when I moved my iPod from one of my host
         computers to another.  But it didn't change when other
         playlist and/or authorization changes changed the 'sess'
         session atom.  The last 128 bytes are all zero-bytes, and
         are ignored during playback.  I was able to randomly change
         the last 128 bytes to anything without affecting playback.
     --- ("guid")
     <ATOM:"guid">
       <uint32> - GUID ID or number (should be three)
       <uint32> - Number of subatoms (should be zero)
       <uint32> - Four zero bytes
       <GUID Data Bytes> - 4 bytes
         This is the final 'guid' atom, the last one, and yes, it's
         only 4 bytes long, containing probably a <uint32> value.
         It has been value six on my iPod.  Changing it broke my
         iPod's ability to play audio files.  I see that the value
         six shows up here, as well in the above 'indx' atom within
         the 'sess' atom.  I can't help but think these two values
         may be related somehow.     
     --- ("guid")
   ===
 --- ("sean")
=== FILE DATA ENDS HERE

WABContactsGroup File

New file I found recently on a 3G iPod. Unknown purpose, although probably related to the new Contacts sync, given the name. Assuming that this may be different for people syncing to Outlook instead of to the Windows Address Book (WAB). The file is 56 bytes in size.

iTunesLock File

This file is created right before iTunes modifies or reads anything on the iPod, and is deleted when it is finished. Does not contain any data, and is probably only used to make sure only one program at a time tries accessing the important files on the iPod at one time.

PhotosFolderName

Lists the watched folders watched and syncronized with the Ipod. Found on the iPod video (probably on the Photo as well). It is a very simple format with the length of the folder name, preceeded by the folder name in unicode letters.

PhotoFolderPrefs

Stores Photo Folder preferences.

PhotoFolderAlbums

Stores Photo Albums .

iTunesPlaylists

Keeps track of the iTunesPlaylists???

iTunesPodcasts

Keeps track of the iTunesPodcasts???

iTunesVideoPlaylists

Keeps track of the iTunesVideoPlaylists???

Preferences

Located at /iPod_Control/Device/Preferences.

I'm unsure what is the best way to parse the timezone information field, but the previous explanation that it's the timezone information in 30 minutes steps is wrong, it doesn't hold up during DST. Something that works is to substract 0x19 (this corresponds to UTC+0) to that value. The resulting value divided by 2 gives the timezone (in hours), and the less significant bit (ie the resulting value modulo 2) indicates if DST is active or not (1 => DST active). When the DST bit is set, 1 hour must be added to the timezone value.

Example:Selecting Paris (DST) will set this offset to 0x1c: (0x1c - 0x19) = 0x03 0x03 / 2 = 1 0x03 % 2 = 1 which gives us a GMT+2 timezone

_volumelocked

Located at /iPod_Control/Device/

Exists when a 4-digit combination has been set for the Volume Limit on 5Gs and nanos (firmware 1.1.1). File length is 0x20. Appears to be encrypted? However, deleting the file removes the volume limit :-)