Much of early literacy takes place at home, when caregivers read books to young children. Previous research demonstrates how engaging in dialogue with children while reading can develop critical reading readiness skills, but most adult readers are unsure if and how to lead effective conversations. We present ContextQ, a tablet-based reading application to unobtrusively present automatically generated dialogic questions to caregivers to support this dialogic reading practice. An ablation study demonstrates how our method of coding educators' experience in the question generation process can produce high-quality results; and through a user study with 12 parent-child dyads (child age: 4-6), we demonstrate that this system can serve as a guide for parents in leading a contextually meaningful dialogue, resulting in significantly more conversational turns of both father and son. child and deeper conversations with connections to the child's daily life.