-x/--extract option for returning content of first fenced code block

[simonw]

Thought of this after writing up my one-shot prompting tricks here: https://simonwillison.net/2024/Dec/19/one-shot-python-tools/

Idea is to be able to do this:

llm 'Python CLI tool for finding all files matching an expression, recursively' --extract

This would run the provided prompt and return just the content of the first fenced code block (```python ...) in the response.

If no fenced code blocks found, returns the whole string.

[simonw]

Used o1-mini for the initial code: https://chatgpt.com/share/67642b13-72a4-8006-8620-f75e62706554

[simonw]

Updated documentation: https://github.com/simonw/llm/blob/67d4a9964501071b3d810a76598ea27cc741ec6f/docs/usage.md#extracting-fenced-code-blocks

[simonw]

Demo:

llm -m gpt-4o-mini 'javascript function to reverse a string' -x

Output:

function reverseString(str) {
    return str.split('').reverse().join('');
}

// Example usage
const originalString = "Hello, World!";
const reversedString = reverseString(originalString);
console.log(reversedString); // Output: "!dlroW ,olleH"

But running llm logs -c shows the full prompt and response: https://gist.github.com/simonw/0964cdd9d7e3cb46932d9b017083f25a

[simonw]

Templates should support this too.

[simonw]

This now works:

llm --system 'write a Python function' --extract --save python-function
llm -t python-function 'reverse a string'

Docs here: https://github.com/simonw/llm/blob/000e984def983aa36384a24df42d4dbb558b5bb1/docs/templates.md#configuring-code-extraction

[simonw]

From my blog entry: https://simonwillison.net/2024/Dec/19/one-shot-python-tools/

llm -s 'You write Python tools as single files. They always start with this comment:

# /// script
# requires-python = ">=3.12"
# ///

These files can include dependencies on libraries such as Click. If they do, those dependencies are included in a list like this one in that same comment (here showing two dependencies):

# /// script
# requires-python = ">=3.12"
# dependencies = [
#     "click",
#     "sqlite-utils",
# ]
# ///' --save python --extract

[simonw]

I tried this:

llm -t python 'starlette server providing /?url= API to extract just text using BeautifulSoup'

And got back this:

# /// script
# requires-python = ">=3.12"
# dependencies = [
#     "starlette",
#     "httpx",
#     "beautifulsoup4",
# ]
# ///

from starlette.applications import Starlette
from starlette.responses import JSONResponse
from starlette.requests import Request
import httpx
from bs4 import BeautifulSoup

app = Starlette()

@app.route("/", methods=["GET"])
async def extract_text(request: Request):
    url = request.query_params.get("url")

    if not url:
        return JSONResponse({"error": "URL parameter is required"}, status_code=400)
    
    try:
        async with httpx.AsyncClient() as client:
            response = await client.get(url)
            response.raise_for_status()
    except httpx.RequestError:
        return JSONResponse({"error": "Failed to fetch the URL"}, status_code=500)
    except httpx.HTTPStatusError as exc:
        return JSONResponse({"error": f"HTTP error occurred: {exc.response.status_code}"}, status_code=exc.response.status_code)

    soup = BeautifulSoup(response.text, 'html.parser')
    text = soup.get_text()

    return JSONResponse({"text": text.strip()})

if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="127.0.0.1", port=8000)

Full transcript here: https://gist.github.com/simonw/b32bb1b9ad3ea36247189f0d66cfc38e

That was with gpt-4o-mini.

[cmosguy]

Hey @simonw how do we do this for o1-preview?

llm -m o1-preview-azure -t python-function 'reverse a string'                                                                                                   

Error: Error code: 400 - {'error': {'message': "Unsupported value: 'messages[0].role' does not support 'system' with this model.", 'type': 'invalid_request_error', 'param': 'messages[0].role', 'code': 'unsupported_value'}}

BTW, I had to do this:

llm -m o1-preview-azure 'You are an expert Python software engineer with over 20 years of experience.  create a script that will collect all the IBM LSF information using the necessary commands to tell me which host is available' -x

That seemed to work ok, but I cannot use templates yet with o1.

[simonw]

That's because o1-preview doesn't support system messages at all. You can't use templates that set system messages with that model.

[cmosguy]

@simonw Yes I suspected that after I wrote that last comment. Thanks for following up on this!