Most of the information on a computer changes very slowly or not at all. This includes the applications themselves, the operating system and even most of the user data. Typically, only a small percentage of the information in a partition or disk changes on a daily, or even a weekly, basis. For that reason, it makes sense only to back up the data that has changed on a daily basis. This is the basis of sophisticated backup strategies.
Differential backups were the next step in the evolution of backup strategies. A differential backup backs up only the files that changed since the last full back. For example, suppose you do a full backup on Sunday. On Monday you back up only the files that changed since Sunday, on Tuesday you back up only the files that changed since Sunday, and so on until the next full backup. Differential backups are quicker than full backups because so much less data is being backed up. But the amount of data being backed up grows with each differential backup until the next full back up. Differential backups are more flexible than full backups, but still unwieldy to do more than about once a day, especially as the next full backup approaches.
Incremental backups also back up only the changed data, but they only back up the data that has changed since the last backup — be it a full or incremental backup. They are sometimes called "differential incremental backups," while differential backups are sometimes called "cumulative incremental backups." Confused yet? Don't be.
I think it's bad practice (see my post on googlification) to link to the first result I found on google for the search "incremental differential". But what the hell! That explanation was satisfactory to me!