I uploaded a corrupted PDF into an LLM to understand how it analyzes it. First, I modified the font using non-contextual alternates and then tested the file across several different models.
As expected, the results are twofold:
- The model uses OCR to read the text. In this case, the human reader and the machine obviously see the same thing.
- The model accesses the text underlying the contextual alternates, thereby reading what would have been written had the font remained unmodified.
This raises critical concerns and a series of questions. Is it possible to perform data injection through this method? What kind of dynamics are enabled between humans and machines when they are not perceiving the same information?