Support for Full Text Search

[dplewis]

Closes: #751

I did run into issues with 27 API Licensing but resolved it.

[codecov]

Codecov Report

Merging #772 into master will increase coverage by 0.03%.
The diff coverage is 100%.

@@             Coverage Diff              @@
##             master     #772      +/-   ##
============================================
+ Coverage     53.38%   53.41%   +0.03%     
- Complexity     1746     1747       +1     
============================================
  Files           132      132              
  Lines         10286    10293       +7     
  Branches       1427     1427              
============================================
+ Hits           5491     5498       +7     
  Misses         4340     4340              
  Partials        455      455

Impacted Files	Coverage Δ	Complexity Δ
Parse/src/main/java/com/parse/ParseQuery.java	74.81% <100%> (+0.33%)	84 <1> (+1)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 60810e9...f66a89c. Read the comment docs.

[dplewis]

@rogerhu Can you look this over?

[rogerhu]

Thanks for adding!

[rogerhu]

FYI - I think the docs need to be improved. Kept getting text index required for $text query error codes even after trying to use the Schema API. I had to manually add the text query to the index even after using the API:

curl -X POST \
  -H "X-Parse-Application-Id: ${APPLICATION_ID}" \
  -H "X-Parse-Master-Key: ${MASTER_KEY}" \
  -H "Content-Type: application/json" \
  -d '
    {
      "className": "GameScore",
      "fields": {
        "playerName": {
          "type": "String"
        }
      },
      "indexes": {
        "indexName": {
          "playerName": 1 
        }
      }
    }' \
  http://localhost:1337/parse/schemas/GameScore

This fixed the problem:

db.GameScore.createIndex({playerName: "text"})

[dplewis]

@rogerhu what version of Parse-server are you running. Text indexes should be automatically created.

[rogerhu]

MongoDB shell version v3.6.2

The Schema API creates an index, but not a text index:

[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "parse.GameScore"
	},
	{
		"v" : 2,
		"key" : {
			"playerName" : 1
		},
		"name" : "indexName",
		"ns" : "parse.GameScore"
	},

Only by calling the Mongo specific index does it work:

	{
		"v" : 2,
		"key" : {
			"_fts" : "text",
			"_ftsx" : 1
		},
		"name" : "playerName_text",
		"ns" : "parse.GameScore",
		"weights" : {
			"playerName" : 1
		},
		"default_language" : "english",
		"language_override" : "language",
		"textIndexVersion" : 3
	}

[rogerhu]

I was also using the latest commit 8ec07b83d0b74b00153b7b414aa73d7247c55f85 of Parse-Server

[rogerhu]

Dropping the table and SCHEMA info and re-creating the table doesn't help either:

> db.GameScore.drop()
true
> db.getCollection("_SCHEMA").remove({"_id": "GameScore"})
WriteResult({ "nRemoved" : 1 })

POST to create some data:

curl -X POST \
  -H "X-Parse-Application-Id: ${APPLICATION_ID}" \
  -H "X-Parse-Master-Key: ${MASTER_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"score":1337,"playerName":"Sean Plott","cheatMode":false}'  \
  http://localhost:1337/parse/schemas/GameScore

Mongo:

> db.getCollection("_SCHEMA").find({"_id": "GameScore"})
{ "_id" : "GameScore", "objectId" : "string", "updatedAt" : "date", "createdAt" : "date" }
> db.GameScore.getIndexes()
[ ]

[rogerhu]

I put a breakpoint but don't ever see createTextIndexesIfNeeded() get called either

[dplewis]

For the schema you could do

 "indexes": {
        "indexName": {
          "playerName": "text"
        }
      }

Thats interesting if createTextIndexesIfNeeded() isn't called. I'll look into it, I was going to improve the indexing after the next parse-server update

[rogerhu]

It seems like I have to explicitly use the "text" field instead of 1 (#772 (comment)):

curl -X POST \
  -H "X-Parse-Application-Id: ${APPLICATION_ID}" \
  -H "X-Parse-Master-Key: ${MASTER_KEY}" \
  -H "Content-Type: application/json" \
  -d '
    {
      "className": "GameScore",
      "fields": {
        "playerName": {
          "type": "String"
        }
      },
      "indexes": {
        "indexName": {
          "playerName": "text"
        }
      }
    }' \
  http://localhost:1337/parse/schemas/GameScore

Then it seems to work:

> use parse
switched to db parse
> db.GameScore.getIndexes()
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "parse.GameScore"
	},
	{
		"v" : 2,
		"key" : {
			"_fts" : "text",
			"_ftsx" : 1
		},
		"name" : "indexName",
		"ns" : "parse.GameScore",
		"weights" : {
			"playerName" : 1
		},
		"default_language" : "english",
		"language_override" : "language",
		"textIndexVersion" : 3
	}
]

Again, it seems like the documentation needs to reflect this aspect. It seems like text indexes get added if you only add new fields to an existing class right?

[rogerhu]

Yep @dplewis that worked.

[dplewis]

@rogerhu I can update the docs. When whereFullText() gets called the server looks for $text inside of createTextIndexesIfNeeded(). Can you debug createTextIndexesIfNeeded

[rogerhu]

Yeah I revised my comment. Create text indexes never gets fired. It seems like it passes in directly the create index command from the dictionary.

[dplewis]

text index required for $text query You shouldn't see this error in the current parse-server 2.7.1

[rogerhu]

That error is coming back from Mongo, not Parse server I believe.

[dplewis]

Ah, I see

[montymxb]

Thanks @dplewis & @rogerhu for getting this in!