A Book of Memories is made up of three first-person narratives:

The first, that of a young Hungarian writer and his fated love for a German poet; we also learn of the narrator's adolescence in Budapest, when he experiences the downfall of his once upper-class but now pro-Communist family.

A second memoir, alternating with the first, is a novel the narrator is composing about a refined Belle Epoque aesthete, whose anti-bourgeois transgressions seem like emotionally overcharged versions of the narrator's own experiences.

A third voice is that of a childhood friend who, after the narrator's return to his homeland, offers an apparently more objective account of their friendship.

Together these brilliantly coloured lives are integrated into a powerful work of tragic intensity.