Need a recipe for annotating consult-line ( consult-grep etc) candidte buffer with which-function

[emacksnotes]

Provide which-function (= "context of the hit") as an annotation to the search hits.

Even without preview turned on, simply having the which-function in completions buffer (either as marginalia annotation or "in place" annotation) would help a lot.

See screenshot for where I imagine the which-function annotation will be displayed. I don't look at the line numbers, and I would be happy if the line numbers of the hits are elided, and which-function is displayed inside.

Recipe for annotating consult-line ( consult-grep etc) with which-function

[minad]

Hi! I am not a which-function user since it turned out to be quite slow in my profiling, but this was years ago. Iirc it repeatedly loads and traverses the Imenu internally, which can be expensive depending on the backend. Nevertheless it would be good if we could ensure that Consult interacts nicely with this mode. There is also the new Breadcrumb as an alternative, which you may want to take a look at?

[minad]

To come back to the annotation question - you can override the :annotate option of consult-line via consult-customize, then extract the location from the candidate string, and subsequently retrieve the name of the function via which-function-mode. This needs quite a bit of hacking though but should be possible.

[emacksnotes]

Hi! I am not a which-function user since it turned out to be quite slow in my profiling, but this was years ago. Ii

which-function is one of the legacy providers ... I am sure lsp and eglot provide context for symbol at point.

it repeatedly loads and traverses the Imenu internally, which can be expensive depending on the backend.

I haven't explored lsp and eglot. It is quite some time since I have used old school TAGS database etc.

If the provider builds a pre-built symbols database, then mapping file + line + column to a context should be an easy job.

(Leaving aside what the provider is and how performant it is, I would really like to hear about what hooks are provided by consult to update the mode line. IOW, if I would like to add a "update mode line" or "do a redisplay" of the preview window, what places I need to hook to. )

I am looking in to some leads, so that I can dig in to the consult codebase to achieve the above task ...

There is also the new Breadcrumb as an alternative, which you may want to take a look at?

Let me see.

[minad]

(Leaving aside what the provider is and how performant it is, I would really like to hear about what hooks are provided by consult to update the mode line. IOW, if I would like to add a "update mode line" or "do a redisplay" of the preview window, what places I need to hook to. )

No such hooks exist so far, since every command needs its own specific preview. However one can use advices. For buffer preview one can advise consult--buffer-preview, and similarly for consult-line preview there is consult--jump-preview.

[kjambunathan]

For a quick draft this works for consult-line

(add-hook 'consult-after-jump-hook 'which-func-update)

---

To the next question ...

1. How do I transform the candidtes (= think "marginalia-like annotation:) in the *completions* buffer, so that it displays the which-function at the point of hit?
2. How can I add this annotation to just the visible set of candidates?

Give me a clue or a starting point, and I can figure out the rest.

[kjambunathan]

When doing consult-grep why is there is NO which-function in mode-line-format?

(Note that my earlier screenshots of consult-line shows that there is INDEED a which-function component in mode-line-format, and it DOES get displayed in the preview window)

---

As a site note, the vc-mode component of mode-line-format of mode-line-format doesn't show up in preview of grep hits but show up in the line hits.

I would assume that consult meddles with mode-line-format and takes a pristine value of mode-line-format (for some definition of pristine)

[kjambunathan]

To the next question ...

1. How do I transform the candidtes  (= _think_ "marginalia-like annotation:)  in the `*completions*` buffer, so that it displays the `which-function` at the point of hit?

2. How can I add this annotation to just the visible set of candidates?

One of the places where it would be helpful, is to look for commands in a 3rd party package.

IMHO, doing M-x consult-line interactive and have the candidates displayed with which-function (= commands) is a much better experience than doing a consult-imenu and search for command markers in the marginalia annotation. See screenshots

Using consult-imenu to quickly identify commmand-s in a Elisp file

(I understand that the way imenu works it is NOT POSSIBLE to get surrounding function context (= command at point).)

As I write this comment, I wonder if this a way for consult-imenu to include command as a separate group in addition to defvar, defcustoms etc.

---

Using M-x consult-line RET interactive RET to identify commands at point

(It appears that having which-function as an annotation in *candidates* buffer would help)

I appreciate any recipe, or quick start hints to accomplish the tasks captured in the screenshots above

[hmelman]

For single buffer functions like consult-line you could probably annotate the lines with the which-function values obtained via imenu or perhaps an lsp though I don't know as either provides a simple "what's the 'function' string for this position" that you could call. You'll probably have to create a function that does what the function which-function does or that calls it while moving point around.

For multi-file functions like consult-grep, the cost of parsing the (possibly many) files is going to much higher than the cost of merely searching them so you're going to want a separate command anyway.

See orderless issue 170 for the current state of annotation matching.

Note also that which-function and imenu are useful in not just elisp buffers as are the mentioned consult commands. So any changes to consult for this would have to work in more contexts (do you want full headers in the annotations for text files?)

[kjambunathan]

1. How do I transform the candidtes (= think "marginalia-like annotation:) in the *completions* buffer, so that it displays the which-function at the point of hit?

which-function as candidate annotation in M-x consult-line output

To get this

or this

use this

(defun consult--candidate-consult-location-which-function (cand)
  (let ((prop 'consult-location))
    (pcase (get-text-property 0 prop cand)
      ((or (and (or (and `(,(and (pred markerp) marker) . ,line)
                         (let `(,buffer . ,point-at-bol)
                           (cons (marker-buffer marker)
                                 (marker-position marker))))
                    `((,buffer . ,point-at-bol) . ,line))
                (let ctx
                  (with-current-buffer buffer
                    (save-excursion
                      (goto-char point-at-bol)
                      (substring-no-properties
                       (or (which-function)
                           ""))))))
           (app (format "%S") ctx))
       ctx))))

(defun consult--candidate-consult-location-lisp (cand)
  (let ((prop 'consult-location))
    (pcase (get-text-property 0 prop cand)
      ((app (format "%S") ctx)
       ctx))))

(defun consult--candidate-consult-location-line-no (cand)
  (pcase (get-text-property 0 'consult-location cand)
    (`(,_ . ,line)
     line)))

(defun consult--line-prefix (&optional curr-line)
  (lexical-let* ((display-style
                  'prefix
                  ;; 'suffix
                  )
                 (consult--candidate-to-context
                  ;; #'consult--candidate-consult-location-line-no
                  #'consult--candidate-consult-location-which-function
                  ;; #'consult--candidate-consult-location-lisp
                  ))
    (pcase display-style
      (`prefix
       (lambda (cand)
         (list cand
               (thread-last cand
                            (funcall (lambda (cand)
                                       (->> cand
                                            (funcall consult--candidate-to-context)
                                            (format "%-50s| ") ; FIXME
                                            (funcall (-rpartial #'propertize 'face 'marginalia-symbol))))))
               "")))
      (`suffix
       (lambda (cand)
         (consult--annotate-align cand
                                  (->> cand
                                       (funcall consult--candidate-to-context)
                                       (funcall (-rpartial #'propertize 'face 'font-lock-warning-face)))))))))

[kjambunathan]

Apropos #993 (comment), I would have liked to see a consult--annotate-align counterpart for "prefix" annotations .

IIUC how consult--annotate-align, works, the position of annotation is made up as candidates are encountered. So, the "padding" is not fixed, but dynamically determined and they may increasing monotonicaly (= a staggered column for annotation) For all practical purposes, going by what appears on the screen, it looks as if the align position is calculated across all candidates.

---

Regarding the performance hit of using which-function (or breadcrumb or whatever other provider there could be), I feel the annotation should happen lazily--what I mean here is that, the annotation may happen as and when candidates are "exposed" on the screen.

So, the bottleneck in "the search and make sense of a 3rd party library" may not the be software that provides the context, but the human who has to exercise his cognitive functions ...

I wonder how consult does annotations ... if it doesn't do annotation lazily (= as and when they are required, and their services are summoned by the user doing a C-v and M-v on the candidates) it could be a good idea to add "on demand" annotations (as opposed to computing them upfront)

[minad]

Yes, annotations are computed lazily.

[kjambunathan]

1. How do I transform the candidtes (= think "marginalia-like annotation:) in the *completions* buffer, so that it displays the which-function at the point of hit?

which-function as candidate annotation in M-x consult-grep output

To get this

FILE + WHICH-FUNCTION as "prefix" annotation of `M-x consult-grep'

or this

FILE + WHICH-FUNCTION as "suffix" annotation of `M-x consult-grep'

do this

diff --git a/consult.el b/consult.el
index ad77917..d74dbe3 100644
--- a/consult.el
+++ b/consult.el
@@ -2557,7 +2557,8 @@ PREVIEW-KEY are the preview keys."
 (cl-defun consult--read-1 (table &key
                                  prompt predicate require-match history default
                                  keymap category initial narrow add-history annotate
-                                 state preview-key sort lookup group inherit-input-method)
+                                 state preview-key sort lookup group inherit-input-method
+				 &allow-other-keys)
   "See `consult--read' for the documentation of the arguments."
   (consult--minibuffer-with-setup-hook
       (:append (lambda ()

and

this

(defun consult--grep-which-function (cand &optional find-file)
  (when cand
    (let* ((file-end (next-single-property-change 0 'face cand))
           (line-end (next-single-property-change (1+ file-end) 'face cand))
           (matches (consult--point-placement cand (1+ line-end) 'consult-grep-context))
           (file (substring-no-properties cand 0 file-end))
           (line (string-to-number (substring-no-properties cand (+ 1 file-end) line-end)))
           ;; Hide line numbers and the separator that follows in candidates
           (_ (add-text-properties (+ 1 file-end ) (+ 1 line-end) 
                                   (list 'invisible t)
                                   cand))
           (buffer (funcall (or find-file #'find-file-noselect) file))
           (column (or (car matches) 0))
           (which-function (when (buffer-live-p buffer)
                             (with-current-buffer buffer
                               (save-excursion
                                 (without-restriction
                                   (goto-char (point-min))
                                   ;; Location data might be invalid by now!
                                   (ignore-errors
                                     (forward-line (1- line))
                                     (goto-char (min (+ (point) column) (pos-eol))))
                                   (point-marker)
                                   (substring-no-properties
                                    (or (which-function) ""))))))))
      (format "%-50s|"
              (format "%s (%s)"
                      (propertize (file-name-nondirectory file) 'face 'marginalia-file-name)
                      (propertize which-function 'face 'marginalia-symbol)
                      )))))

;; To push an `:annotate' property to `consult-grep' command, do
;; `&allow-other-keys' in `consult--read-1'
(consult-customize consult-grep
                   :annotate
                   (lambda (cand)
                     (let ((display-style
                            'prefix
                            ;; 'suffix
                            ))
                       (pcase display-style
                         (`suffix
                          (consult--annotate-align cand
                                                   (consult--grep-which-function cand)))
                         (`prefix
                          (list cand (consult--grep-which-function cand) ""))))))

[kjambunathan]

Apropos #993 (comment) , how open you will be in adding &allow-other-key to consult--read-1.

IOW, I need some way to push an :annotate property to consult-grep (or possibly other commands).

If the "push new property" is in place for :annotate then I don't have to patch consult.el.

(Note that I am the author of ox-odt exporter in stock Emacs, I once had copyright assignment with FSF which covered "future" contributions, but it is almost a decade since I have revoked "further" contributions to FSF)

[minad]

No, we cannot do that, since it will make the error checking of the API worse.

[kjambunathan]

No, we cannot do that, since it will make the error checking of the API worse.

Can you think of any other way? I just don't want to patch consult.el.

How about adding :annotate as a dummy property of allowable keys in consult-grep?

(You know what I mean)

[minad]

I don't understand why you need allow-other-keys. The key :annotate is already supported by consult--read and consult--read-1. Using consult-customize should already work as is.

[kjambunathan]

Apropos #993 (comment), I would have liked to see a consult--annotate-align counterpart for "prefix" annotations .

IIUC how consult--annotate-align, works, the position of annotation is made up as candidates are encountered. So, the "padding" is not fixed, but dynamically determined and they may increasing monotonicaly (= a staggered column for annotation) For all practical purposes, going by what appears on the screen, it looks as if the align position is calculated across all candidates.

Do you think there is a scope for improvement in how prefixes and suffixes are "aligned"?

Screenshots created with recipe shared in #993 (comment)

(The screenshots below are self-explanatory)

Any way prefix annotation could be aligned nicely?

Any way suffix annotations could behave reasonably well in the following case?

(Artefact 1) and ...

... (Artefact 2)

[minad]

Do you think there is a scope for improvement in how prefixes and suffixes are "aligned"?

It shouldn't be difficult to compute the alignment directly in the annotation function such that strings of a constant width are returned. You can use a variable which is updated when new annotations are computed:

(defvar my-prefix-width 0)
(propertize " " 'display '(:align-to (+ left my-prefix-width)))

[kjambunathan]

To get this

FILE + WHICH-FUNCTION as "prefix" annotation of `M-x consult-grep'

(Note that the FILE + WHICH-FUNCTION annotation and the SEARCH HITS always start at the same column)

do this

diff --git a/consult.el b/consult.el
index ad77917..134ddee 100644
--- a/consult.el
+++ b/consult.el
@@ -2557,7 +2557,8 @@ PREVIEW-KEY are the preview keys."
 (cl-defun consult--read-1 (table &key
                                  prompt predicate require-match history default
                                  keymap category initial narrow add-history annotate
-                                 state preview-key sort lookup group inherit-input-method)
+                                 state preview-key sort lookup group inherit-input-method
+				 &allow-other-keys)
   "See `consult--read' for the documentation of the arguments."
   (consult--minibuffer-with-setup-hook
       (:append (lambda ()
@@ -4728,6 +4729,11 @@ BUILDER is the command line builder function."
                     (setq content (substring content 0 consult-grep-max-columns)))
                   (when highlight
                     (funcall highlight content))
+
+		  ;; Hide leading white space in file text
+		  (when (string-match (rx (and bos (one-or-more (any " " "\t")))) content)
+		    (add-text-properties 0 (match-end 0) '(invisible t) content))
+
                   (setq str (concat file sep line sep content))
                   ;; Store file name in order to avoid allocations in `consult--prefix-group'
                   (add-text-properties 0 file-len `(face consult-file consult--prefix-group ,file) str)

and this

;; `which-function' annotation for `M-x consult-grep'

(custom-set-variables
 `(vertico-count 5))

(custom-set-faces
 '(consult-grep-context ((t ( :background "whitesmoke"
                              :box (:line-width (2 . 2) :color "grey75" :style released-button)
                              :height 1.01
                              )))))

(defface consult-annotate-face
  '((t (
        :box ( :line-width (2 . 2)
               :style flat-button
               )
        :overline t
        :weight bold
        :height 1.01
        )))
  "Annotate face."
  :group 'faces)

(defface consult-annotate-file-name-face
  '((t ( :inherit (consult-annotate-face font-lock-function-name-face))))
  "Annotate file name face."
  :group 'faces)

(defface consult-annotate-which-function-face
  '((t ( :inherit (consult-annotate-face font-lock-comment-face))))
  "Annotate which function face."
  :group 'faces)

(defface consult-annotate-indent-face
  '((t ( :background "gray97")))
  "Annotate indent face."
  :group 'faces)

(defun consult--grep-which-function (cand &optional find-file)
  (when cand
    (let* ((file-end (next-single-property-change 0 'face cand))
           (line-end (next-single-property-change (1+ file-end) 'face cand))
           (matches (consult--point-placement cand (1+ line-end) 'consult-grep-context))
           (file (substring-no-properties cand 0 file-end))
           (line (string-to-number (substring-no-properties cand (+ 1 file-end) line-end)))
           ;; Hide line numbers and the separator that follows in candidates
           (_ (add-text-properties (+ 1 file-end) (+ 1 line-end)
                                   (list 'invisible t)
                                   cand))
           (buffer (funcall (or find-file #'find-file-noselect) file))
           (column (or (car matches) 0))
           (which-function (when (buffer-live-p buffer)
                             (with-current-buffer buffer
                               (save-excursion
                                 (without-restriction
                                   (goto-char (point-min))
                                   ;; Location data might be invalid by now!
                                   (ignore-errors
                                     (forward-line (1- line))
                                     (goto-char (min (+ (point) column) (pos-eol))))
                                   (point-marker)
                                   (substring-no-properties
                                    (or (which-function) ""))))))))
      (concat
       (propertize (format "%s" which-function) 'face 'consult-annotate-which-function-face)
       (propertize (format " [%s]" (file-name-nondirectory file)) 'face 'consult-annotate-file-name-face)))))

;; To push an `:annotate' property to `consult-grep' command, do
;; `&allow-other-keys' in `consult--read-1'
(consult-customize consult-grep
                   :annotate
                   (lambda (cand)
                     (list
                      ;; Candidate
                      (progn
                        (add-face-text-property 0 (length cand) 'consult-grep-context 'append cand)
                        cand)
                      ;; Prefix
                      (format "%s\n%s"
                              (consult--grep-which-function cand)
                              (propertize "\t\t" 'face 'consult-annotate-indent-face))
                      ;; Suffix
                      ""))
                   )

[kjambunathan]

In iteration shown at #993 (comment), the significant improvement is

• the "prefix annotation" is followed by a newline. This way the context stands apart from the search hit
• the search hits start at the same column position. This is achieved by hiding the leading whitespace in the matched text.

FWIW, I like this implementation. The only drawback is the number of hits (= vertico-count) is halved because of the newline that follow the prefix.

[minad]

the "prefix annotation" is followed by a newline. This way the context stands apart from the search hit

The completion UIs usually do not expect prefix/suffix annotations with newlines.