Tweaking LLM responses

Formatting in-buffer refactored code

Depending on your model or prompt, sometimes returned in-buffer refactored code would not be perfect, i.e. it would have incorrect indentation or even include Markdown formatting. The below is an example for how to add a hook to automatically fix such cases.

(cl-defun my/clean-up-gptel-refactored-code (beg end)
  "Clean up the code responses for refactored code in the current buffer.

The response is placed between BEG and END.  The current buffer is
guaranteed to be the response buffer."
  (when gptel-mode          ; Don't want this to happen in the dedicated buffer.
    (cl-return-from my/clean-up-gptel-refactored-code))
  (when (and beg end)
    (save-excursion
      (let ((contents
             (replace-regexp-in-string
              "\n*``.*\n*" ""
              (buffer-substring-no-properties beg end))))
        (delete-region beg end)
        (goto-char beg)
        (insert contents))
      ;; Indent the code to match the buffer indentation if it's messed up.
      (indent-region beg end)
      (pulse-momentary-highlight-region beg end))))

Then, where you config gptel, add:

(add-hook 'gptel-post-response-functions #'my/clean-up-gptel-refactored-code)

What this does is remove the Markdown tags and automatically indent the code with respect to the buffer. When you send the prompt, you can see the streamed-in bad code, and after the streaming is complete, it refactors it.

Rendering LaTeX in dedicated chat buffer

Sometimes the chat model likes to talk in LaTeX when you ask it math questions. Most online LLM interfaces support rendering of LaTeX through MathJax. Below is some code that will configure Org mode to allow rendering LaTeX, even if your GPTel buffer does not use Org mode. This is achieved by switching to Org, rendering the LaTeX (if any are detected) and then switching back to the previous mode.

This is quite experimental, and might have some side effects with certain modes.

;; If you already have an Org config, then you need to adjust it so that it has
;; the below adjustments.
(use-package org
  :config
  ;; Adjust the colors to fit the Org buffer. This only applies if your
  ;; dedicated GPTel buffer is Org mode.
  (plist-put org-format-latex-options :foreground nil)
  (plist-put org-format-latex-options :background nil)
  ;; Use a different process for generating LaTeX so that we can scale it.
  (customize-set-variable 'org-preview-latex-default-process 'dvisvgm)
  ;; Make it so that the LaTeX scales when the text scale is adjusted.
  (add-hook 'org-mode-hook (lambda ()
                             (add-hook 'text-scale-mode-hook
                                       (lambda ()
                                         (my/resize-org-latex-overlays)
                                         (my/adjust-org-latex-overlay-padding))
                                       nil t))))

;; If you are using Markdown for your dedicated GPTel buffer, then this is
;; required.
(use-package markdown
  :config
  (add-hook 'markdown-mode-hook
            (lambda ()
              (add-hook 'text-scale-mode-hook
                        (lambda ()
                          ;; Even though this is for Org mode, we can use it in
                          ;; other buffers.
                          (my/resize-org-latex-overlays)
                          (my/adjust-org-latex-overlay-padding))
                        nil t))))

;; Add the hook so that the below functions will run once the LLM response has
;; finished.
(add-hook 'gptel-post-response-functions #'my/clean-up-llm-response)

(defun my/adjust-org-latex-overlay-padding ()
  (save-excursion
    (goto-char (point-min))
    (while (search-forward "\\[" nil t)
      (let* ((o (car (overlays-at (point))))
             (display-plist (cdr (overlay-get o 'display))))
        (when (eq (overlay-get o 'org-overlay-type) 'org-latex-overlay)
          (plist-put display-plist :margin
                     (cons 0 (default-line-height))))))))

(defun my/resize-org-latex-overlays ()
  (defvar text-scale-mode-step)
  (defvar text-scale-mode-amount)
  (cl-loop for o in (car (overlay-lists))
           if (eq (overlay-get o 'org-overlay-type) 'org-latex-overlay)
           do (plist-put (cdr (overlay-get o 'display))
		         :scale (expt text-scale-mode-step
				      text-scale-mode-amount))))

(defun my/clean-up-llm-chat-response (beg end)
  (when (save-excursion
          (goto-char beg)
          (re-search-forward (rx (or (group "\\" (zero-or-more any) "\\")
                                     (group "[" (zero-or-more any) "]")))
                             end t))
    (goto-char beg)
    (let ((original-mode major-mode)
          (text-scale-amount text-scale-mode-amount))
      (unless (eq major-mode 'org-mode)
        (org-mode))
      (org-latex-preview)
      (unless (eq original-mode 'org-mode)
        (funcall original-mode)
        (gptel-mode)
        (text-scale-set text-scale-amount))
      (my/resize-org-latex-overlays)
      (my/adjust-org-latex-overlay-padding)
      (pulse-momentary-highlight-region beg end))))

(defun my/clean-up-llm-response (beg end)
  (if (bound-and-true-p gptel-mode) ; Make sure we are in the dedicated buffer.
      (my/clean-up-llm-chat-response beg end))