Screenshots, scans and photographs of paper

What a screenshot buys you

Vision-capable models let you paste a picture: a screenshot of an error, a photo of a paper form, a whiteboard after a meeting, a chart from a report, a picture of a damaged part.

For some office work this is genuinely transformative. Explaining an unfamiliar error dialogue, transcribing a whiteboard before it is wiped, getting the text off a printed letter you have no digital copy of, working out which field of a government form is being asked about — all of these were previously "type it out yourself" and are now thirty seconds.

For other work it fails in a specific way that is worth learning, because the failure looks identical to success.

Reading a chart is not reading a chart

Ask for the values in a bar chart where the numbers are not printed on the bars, and you will get numbers. They will be plausible, in the right order, roughly the right shape.

They are estimates. The model reads the title, the axis labels and the visual proportions, and produces values consistent with that. It is not measuring pixels against an axis with any precision, and nothing in the reply distinguishes "this number was printed on the chart" from "this number is my impression of that bar's height".

The rule: if the number is not written in the image, treat any number you get back as an approximation, however precisely it is stated. And if you have the underlying data, use the data. Photographing a chart to ask what it says is throwing away the numbers and then asking for them back.

The other reliable weaknesses

Handwriting, especially digits. Ones and sevens, threes and eights, zeros and sixes, and the European crossed seven. A handwritten figure transcribed from a photo is never to be used without checking the original, and this includes your own notes.

Counting. How many items in this photo, how many rows in this table, how many people in this room. Counting is weak in general and weaker in images.

Dense tables in screenshots. Rows drift, columns merge, and a value from the row above lands in the row below. If the table matters, get the underlying file.

Small or low-resolution text. The model will produce a plausible word rather than report that it cannot read one — the same mechanism as everywhere else, applied to pixels.

The instructions that help

Use the same discipline as text extraction:

Transcribe the text in this image exactly. If any character is unclear, write ? in its place. Do not correct spelling, do not complete abbreviations, do not infer anything that is not legible.

Then, as a separate step, ask for the analysis. Evidence before verdict, exactly as before — and now you have a transcription you can read against the original.

Practically: crop to the region you care about rather than sending the whole screen, send one page per image rather than four pages at a distance, and photograph flat with the light behind you. Resolution matters more than anything else in the prompt.

What else is in the picture

An image carries more than you meant to send, and this is the most commonly missed confidentiality failure in the whole course.

Metadata. Photographs from a phone typically carry EXIF data: GPS coordinates, timestamp, device. A photo of a document taken in a client's office records where the client's office is.

Everything in the frame. The colleague's monitor behind the whiteboard. The patient's wristband at the edge of the shot. The next patient's name on the list under the form. The sticky note with a password. The whole image goes to the provider, not the part you were looking at.

Crop first, then strip metadata. Free tools do this completely: exiftool -all= photo.jpg removes it from the command line, GIMP lets you export without metadata, and most phones have a "remove location" option in the share sheet. Cropping in a viewer that only hides the edge is not cropping — export a new file.

The free path for text on paper

For getting text off documents, a dedicated OCR tool is often better than a vision model, and always more honest.

Tesseract is free, open-source, runs offline and handles over 100 languages. ocrmypdf wraps it to add a searchable text layer to a scanned PDF in one command. Neither invents anything: where the scan is bad you get garbage characters, which look like garbage and cannot be mistaken for a reading.

That is the decisive advantage for anything numeric. A vision model gives you a clean, confident, possibly wrong figure. Tesseract gives you 1O42 and you know to go and look. For an invoice total, the ugly honest answer is the better tool, and you can send the OCR output to a model afterwards to tidy it up — with the original in front of you.