Readings & Meanings:
My diary was surreptitiously read by him.
I signed the report.
I don't have a clear memory of whether or not I said that.
言, Gon-ben, is the radical.
己 is the phonetic, which is a pictograph of a pattern found on ancient pottery, originally meaning "arise" but came to mean "oneself."
If Kanji Networks is right about the meaning of 己 in 記, then the idea is "words that help raise clues/signs." An alternative idea would be that the words you write down are self-reflective.