Not able to do proper path monitoring or querying with Firebase and ElasticSearch (with Flashlight) : <p>I'm trying to integrate ElasticSearch wit Firebase. I'm using the <a href="https://github.com/firebase/flashlight" rel="nofollow">Flashlight integration</a> from Firebase to set ut all up. I have deployed the code to Heroku as described in the Github repo linked above.</p>

<p>It works in the sense that when I insert a query objet to <code>/search/request/</code> I get a <code>/search/response</code> result. But the results are a bit messy, not correct. But I can't figure out what is wrong.</p>

<p>This is what is defined in <code>config.js</code> where I define the paths to monitor:</p>

<pre><code>/** Paths to Monitor
 *
 * Each path can have these keys:
 * {string}   path:    [required] the Firebase path to be monitored, for example, `users/profiles`
 *                     would monitor https://&lt;instance&gt;.firebaseio.com/users/profiles
 * {string}   index:   [required] the name of the ES index to write data into
 * {string}   type:    [required] name of the ES object type this document will be stored as
 * {Array}    fields:  list of fields to be monitored and indexed (defaults to all fields, ignored if "parser" is specified)
 * {Array}    omit:    list of fields that should not be indexed in ES (ignored if "parser" is specified)
 * {Function} filter:  if provided, only records that return true are indexed
 * {Function} parser:  if provided, the results of this function are passed to ES, rather than the raw data (fields is ignored if this is used)
 *
 * To store your paths dynamically, rather than specifying them all here, you can store them in Firebase.
 * Format each path object with the same keys described above, and store the array of paths at whatever
 * location you specified in the FB_PATHS variable. Be sure to restrict that data in your Security Rules.
 ****************************************************/

exports.paths = [{
    path: "users",
    index: "firebase",
    type: "user"
}, {
    path: "messages",
    index: "firebase",
    type: "message",
    fields: ['msg', 'name'],
    filter: function(data) {
        return data.name !== 'system';
    }
}];
</code></pre>

<p>This is the structure of the user node in Firebase (it's located at the root).</p>

<pre><code>"users" : {
    "userid123" : {
      "friends" : {
        "userid42" : 1
      },
      "recs" : {
        "-recid1234" : 0
      },
      "user_info" : {
        "createdAt" : 1475157596,
        "email" : "e@male.org",
        "name" : "Firstname Lastname Johnson",
        "profilePicture" : "png.jpg",
        "pushId" : {
        },
        "username" : "userLooser"
      }
    },
    "userid42" : {
      "friends" : {
        "userid123" : 1,
        "test1" : 1
      },
      "recs" : {
        "-recid5678" : 0
      },
      "user_info" : {
        "email" : "email@icloud.com",
        "name" : "Firstname Lastname",
        "phoneNumber" : "",
        "profilePicture" : "jpg.png",
        "pushId" : {

        },
        "username" : "userName"
      }
    }
  }
}
</code></pre>

<p>What I want to achieve is to search for username, and return the top results. All i really need in return is the username and user id's.</p>

<p><a href="https://i.stack.imgur.com/9iIDi.png" rel="nofollow"><img src="https://i.stack.imgur.com/9iIDi.png" alt="Image of firebase search query"></a></p>

<p>But I get the response below, when I query with the object in the image above:</p>

<pre><code>  "search" : {
    "response" : {
      "index" : {
        ".priority" : 1.477326417519E12,
        "hits" : [ {
          "_id" : "test1",
          "_index" : "firebase",
          "_score" : 1,
          "_source" : {
            "friends" : {
              "test1" : 1
            },
            "recs" : {
            },
            "user_info" : {
              "createdAt" : 1475157596,
              "name" : "Testbruker 1",
              "username" : "testuzer"
            }
          },
          "_type" : "user"
        }, {
          "_id" : "userid123",
          "_index" : "firebase",
          "_score" : 1,
          "_source" : {
            "friends" : {
            },
            "recs" : {

            },
            "user_info" : {
              "email" : "",
              "name" : "Firstname Lastname",
              "phoneNumber" : "",
              "profilePicture" : "",
              "pushId" : {
              },
              "username" : "profileName"
            }
          },
          "_type" : "user"
        }, {
          "_id" : "userid42",
          "_index" : "firebase",
          "_score" : 1,
          "_source" : {
            "friends" : {
            },
            "recs" : {

            },
            "user_info" : {
              "createdAt" : 1475157596,
              "email" : "e@male.org",
              "name" : "Firstname Lastname Johnson",
              "profilePicture" : "",
              "pushId" : {
              },
              "username" : "userLooser"
            }
          },
          "_type" : "user"
        } ],
        "max_score" : 1,
        "total" : 3
      },
      "query" : {
        ".priority" : 1.477326417484E12,
        "hits" : [ {
          "_id" : "test1",
          "_index" : "firebase",
          "_score" : 1,
          "_source" : {
            "friends" : {
              "test1" : 1
            },
            "recs" : {
            },
            "user_info" : {
              "createdAt" : 1475157596,
              "name" : "Testbruker 1",
              "username" : "testuzer"
            }
          },
          "_type" : "user"
        }, {
          "_id" : "userid42",
          "_index" : "firebase",
          "_score" : 1,
          "_source" : {
            "friends" : {

            },
            "recs" : {

            },
            "user_info" : {
              "email" : "",
              "name" : "Firstname Lastname",
              "phoneNumber" : "",
              "profilePicture" : "",
              "pushId" : {

              },
              "username" : "profileName"
            }
          },
          "_type" : "user"
        }, {
          "_id" : "userid123",
          "_index" : "firebase",
          "_score" : 1,
          "_source" : {
            "friends" : {
            },
            "recs" : {

            },
            "user_info" : {
              "createdAt" : 1475157596,
              "email" : "e@male.org",
              "name" : "Firstname Lastname Johnson",
              "profilePicture" : "",
              "pushId" : {

              },
              "username" : "userLooser"
            }
          },
          "_type" : "user"
        } ],
        "max_score" : 1,
        "total" : 3
      },
      "type" : {
        ".priority" : 1.477326417503E12,
        "hits" : [ {
          "_id" : "test1",
          "_index" : "firebase",
          "_score" : 1,
          "_source" : {
            "friends" : {
              "test1" : 1
            },
            "recs" : {
            },
            "user_info" : {
              "createdAt" : 1475157596,
              "name" : "Testbruker 1",
              "username" : "testuzer"
            }
          },
          "_type" : "user"
        }, {
          "_id" : "userid42",
          // Same content as above
        }, {
          "_id" : "userid123",
          // Same content as above
        } ],
        "max_score" : 1,
        "total" : 3
      }
    }
  }
</code></pre>

<p>Which:</p>

<ul>
<li><strong>A)</strong> Does not return the correct sorting. The first result didn't match my search query at all, it just looks like it returns all users in random order.</li>
<li><strong>B)</strong> The response is really messy. under <code>/response</code> there is three nodes <code>query</code>, <code>type</code> and <code>index</code> all containing lot's of hits. </li>
</ul>

<p>So, anyone know what's wrong? How can I create a query, or craft the path monitor so the search will work? And maybe get a simpler response?</p>
