Painless: BWC breaking parsing change in 6.3.0 onwards

[spinscale]

Elasticsearch version (bin/elasticsearch --version): 6.4.0/6.3.2 (works in 6.2.4)

Description of the problem including expected versus actual behavior:

Parsing of scripts seems to have changed from 6.2 onwards, meaning that requests and watches that worked on 6.2 will suddenly break in 6.3 and above.

Steps to reproduce:

Run this script field with 6.3 and you will get

GET _search
{
  "script_fields": {
    "test": {
      "script": {
        "source" : "def x = 1; if (x == 1) return true"
      }
    }
  }
}

This is the response

{
  "error": {
    "root_cause": [
      {
        "type": "script_exception",
        "reason": "compile error",
        "script_stack": [
          "... ; if (x == 1) return true",
          "                             ^---- HERE"
        ],
        "script": "def x = 1; if (x == 1) return true",
        "lang": "painless"
      }
    ],
    "type": "search_phase_execution_exception",
    "reason": "all shards failed",
    "phase": "query",
    "grouped": true,
    "failed_shards": [
      {
        "shard": 0,
        "index": ".kibana",
        "node": "R2u0OqZuReaW5_cuIiso7w",
        "reason": {
          "type": "script_exception",
          "reason": "compile error",
          "script_stack": [
            "... ; if (x == 1) return true",
            "                             ^---- HERE"
          ],
          "script": "def x = 1; if (x == 1) return true",
          "lang": "painless",
          "caused_by": {
            "type": "illegal_argument_exception",
            "reason": "unexpected token ['<EOF>'] was expecting one of [';']."
          }
        }
      }
    ]
  },
  "status": 500
}

This above search will work fine under 6.2.4 though. If you append a semicolon at the end of the script, this also works in 6.3.2/6.4.0.

@jdconrad can you take a look please? This is breaking BWC compat in a minor release thus I think it warrants being a blocker for now.

[elasticmachine]

Pinging @elastic/es-core-infra

[jdconrad]

@spinscale Will take a look soon.

[jdconrad]

Posted a fix (#33212)

[jdconrad]

This is fixed.