We introduce a method to automatically improve character models for a handwritten script without the use of transcriptions and using a minimum of document speciļ¬c training data. We show that we can use searches for the words in a dictionary to identify portions of the document whose transcriptions are unambiguous. Using templates extracted from those regions, we retrain our character prediction model to drastically improve our search retrieval performance for words in the document.