Introduction:

The goal of this document is to provide an overview of the various parts/files

involved when libgpod interacts with a device. Ideally it will be helpful to

help application writers/distributors/... figure out what's going wrong when 

trying to interact with an iPod-like device.

Please note that not all devices need all these steps (especially the

hash58/hash72 stuff and the sqlite databases), see the table at the end for

more details

Overview:

* device is plugged in

HAL/udev detects the insertion and runs a callout. This callout sends the

appropriate commands (raw SCSI/USB commands or special AFC command) to the

device to get an XML file describing the device capabilities (artwork formats

supported, serial number, ...) and dumps this XML file to

iPod_Control/Device/SysInfoExtended for future use by libgpod. libgpod doesn't 

do it directly because sending these SCSI/USB commands might need elevated

priviledges.

* the application uses libgpod to read the device content

iPod_Control/iTunes/iTunesDB is accessed as well as a few other files. If

the device needs an hash72 (either in its iTunesDB or because it's using sqlite

databases), then the necessary information is extracted from the existing 

iTunesDB to generate an iPod_Control/Device/HashInfo file (if it doesn't

exist) which will be useful to generate any hash72 we need for this device.

* the application uses libgpod to write to the device

libgpod generates an iPod_Control/iTunes/iTunesDB file. If an hash58 is needed

then the device FirewireID is used to generate it. If an hash72 is needed

then the HashInfo file is used to generate it. If the device uses sqlite

databases, they are generated as well. Post process SQL commands are then 

extracted from the device (either by getting them from SysInfoExtended or by 

getting them through AFC) and executed after the sqlite database generation. 

These post process commands are useful to make sure the sqlite databases layout

matches what the current device firmware expects.

Glossary:

iTunesDB: binary file containing information about all the songs, playlists, ...

stored on the device. On recent devices (iPhoneOS 3.x, Nano 5g), it's replaced

by iTunesCDB which is a compressed version of iTunesDB.

sqlite: recent devices (iPhoneOS 3.x, Nano5g) use sqlite databases instead

of an iTunesDB to store song information, playlists, ... More accurately, they 

have both an iTunesCDB (compressed iTunesDB) used by iTunes to know the device

content and sqlite databases used by the device. Along with these sqlite

databases, there's a .cbk file which contains checksum information and a hash72

for the sqlite data.

iPhoneOS: umbrella for iPhone-like devices, ie iPhones, iPod Touch and (likely)

iPad. On these devices, what matters is the firmware version, not the device

type.

hash58: first iTunesDB hashing scheme introduced by Apple. It appeared in the 

iPod Nano Video (3g)  and iPod Classic and was used by iPhoneOS 1.x. It's fully

reverse-engineered and uses the iPod FirewireID (called this way even on USB

devices) as part of the calculation.

hash72: hashing scheme that was first introduced in iPhoneOS 2.x and has then

been used on the iPod Nano with a camera (5g). It was much more complicated to 

reverse engineer and is not yet 100% known. However, given a file with a valid

hash72, we can extract some hashing data to be able to generate valid hash72

hashes for any file. 

iPod feature matrix:

		SysInfoExtended   hash58   hash72   sqlite   iTunesCDB

iPod 1G	                     no       no       no       no          no

iPod 2G	                     no       no       no       no          no

iPod 3G	                     no       no       no       no          no

Mini 1G	                     no       no       no       no          no

Mini 2G	                     no       no       no       no          no

iPod 4G                     yes       no       no       no          no

iPod 5G                     yes       no       no       no          no

Nano 1G                     yes       no       no       no          no

Nano 2G                     yes       no       no       no          no

iPod Classic                yes      yes       no       no          no

Nano3G                      yes      yes       no       no          no

Nano4G                      yes      yes       no       no          no

iPhoneOS 1.x                yes      yes       no       no          no

iPhoneOS 2.x                yes       no      yes       no          no

Nano5G                      yes   yes[1]      yes      yes         yes

iPhoneOS 3.x                yes       no      yes      yes         yes

[1] surprisingly, the Nano5G uses a hash58 for its iTunesCDB and a hash72 

for its sqlite cbk file. iPhoneOS 3.x uses hash72 everywhere.