Careeroza

Images and Vision

Sending Images

Vision models accept images (JPEG, PNG, GIF, WebP) as image content blocks, from base64 data or a URL. Useful for: photos of damaged products, screenshots of payment errors, receipts and invoices, product photos for catalogue tagging, handwritten notes.

import base64

with open("damaged_parcel.jpg", "rb") as f:
    img = base64.standard_b64encode(f.read()).decode("utf-8")

response = client.messages.create(
    model="claude-sonnet-5-5",
    max_tokens=500,
    messages=[{
        "role": "user",
        "content": [
            {"type": "image", "source": {"type": "base64", "media_type": "image/jpeg", "data": img}},
            {"type": "text", "text": "A customer sent this photo with a damage claim. Describe the visible damage "
                                     "and say whether the product itself looks damaged or only the box."},
        ],
    }],
)
print(next(b.text for b in response.content if b.type == "text"))

# Or from a public URL:
# {"type": "image", "source": {"type": "url", "url": "https://cdn.shopease.in/p/123.jpg"}}

Run it

python damage_check.py

Example output

The outer carton is crushed at one corner and the tape is torn. The steel bottle inside has a visible dent near the base and a scratch along the side, so the product itself appears damaged, not just the box. Recommend: approve replacement.

Detecting the Image Type

Example

The media_type must match the real file. Checking the first bytes ("magic numbers") is more reliable than trusting a file name a user uploaded:

SIGNATURES = [
    (b"\x89PNG\r\n\x1a\n", "image/png"),
    (b"\xff\xd8\xff", "image/jpeg"),
    (b"GIF89a", "image/gif"),
    (b"GIF87a", "image/gif"),
    (b"%PDF-", "application/pdf"),
]

def media_type(data):
    for sig, mt in SIGNATURES:
        if data.startswith(sig):
            return mt
    if data[:4] == b"RIFF" and data[8:12] == b"WEBP":
        return "image/webp"
    return None

samples = {
    "photo.jpg": b"\xff\xd8\xff\xe0" + b"\x00" * 20,
    "screenshot.png": b"\x89PNG\r\n\x1a\n" + b"\x00" * 20,
    "banner.webp": b"RIFF\x00\x00\x00\x00WEBPVP8 ",
    "invoice.jpg": b"%PDF-1.7 ...",                     # wrong extension!
    "virus.png": b"MZ\x90\x00",
}
for name, data in samples.items():
    print(f"{name:15} -> {media_type(data) or 'rejected: unsupported file'}")

Output

photo.jpg       -> image/jpeg
screenshot.png  -> image/png
banner.webp     -> image/webp
invoice.jpg     -> application/pdf
virus.png       -> rejected: unsupported file

Note: Images cost input tokens roughly in proportion to their pixel size. Resize big phone photos (e.g. to about 1500 px on the long side) before sending, unless fine detail matters.