x-pack/filebeat/input/cel: enhance error status reporting in error co…

…nditions (#40083)

We currently update the input status for single object events, but the error
message held by the event is not reported via the UI. We have that value
available, so use it to report the error, but do not include any other parts
of the state as it may leak secrets.

CHANGELOG.next.asciidoc

@@ -297,6 +297,7 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
 - Add ability to remove request trace logs from http_endpoint input. {pull}40005[40005]
 - Add ability to remove request trace logs from entityanalytics input. {pull}40004[40004]
 - Relax constraint on Base DN in entity analytics Active Directory provider. {pull}40054[40054]
+- Enhance input state reporting for CEL evaluations that return a single error object in events. {pull}40083[40083]
 
 *Auditbeat*
 

x-pack/filebeat/docs/inputs/input-cel.asciidoc

@@ -134,7 +134,7 @@ After completion of a program's execution it should return a single object with
 ----
 
 <1> The `events` field must be present, but may be empty or null. If it is not empty, it must only have objects as elements.
-The field should be an array, but in the case of an error condition in the CEL program it is acceptable to return a single object instead of an array; this will will be wrapped as an array for publication and an error will be logged.
+The field should be an array, but in the case of an error condition in the CEL program it is acceptable to return a single object instead of an array; this will will be wrapped as an array for publication and an error will be logged. If the single object contains a key, "error", the error value will be used to update the status of the input to report to Elastic Agent. This can be used to more rapidly respond to API failures.
 
 <2> If `cursor` is present it must be either be a single object or an array with the same length as events; each element _i_ of the `cursor` will be the details for obtaining the events at and beyond event _i_ in the `events` array. If the `cursor` is a single object it is will be the details for obtaining events after the last event in the `events` array and will only be retained on successful publication of all the events in the `events` array.
 

x-pack/filebeat/input/cel/input.go

@@ -382,7 +382,11 @@ func (i input) run(env v2.Context, src *source, cursor map[string]interface{}, p
 					return nil
 				}
 				log.Errorw("single event object returned by evaluation", "event", e)
-				env.UpdateStatus(status.Degraded, "single event object returned by evaluation")
+				if err, ok := e["error"]; ok {
+					env.UpdateStatus(status.Degraded, fmt.Sprintf("single event error object returned by evaluation: %s", mapstr.M{"error": err}))
+				} else {
+					env.UpdateStatus(status.Degraded, "single event object returned by evaluation")
+				}
 				isDegraded = true
 				events = []interface{}{e}
 				// Make sure the cursor is not updated.