By Big_Mac


2016-03-11 03:11:32 8 Comments

I have an array of keys which lead to post objects for my social network like so /posts/id/(post info)

When I load the posts I load /posts/0 and then /posts/1 etc using the observeSingleEventOfType(.Value) method.

I use a lazyTableView to load 30 at a time and it is quite slow. Is there any way I can use one of the query methods or another way of making it faster even if I have to restructure the data in my JSON tree.

I am coming from Parse re-implementing my app and so far the experience as been quite good. Just this one thing I am a bit stuck on. Thanks in advance for the help!

EDIT:

func loadNext(i: Int) { 

    // check if exhists
    let ideaPostsRef = Firebase(url: "https://APPURL")

    ideaPostsRef.childByAppendingPath(i.description).observeSingleEventOfType(.Value, withBlock: {
        (snapshot) in

        if i % 29 == 0 && i != 0 && !self.hitNull { return }
            // false if nil
            // true if not nil
        if !(snapshot.value is NSNull) {
            let postJSON  = snapshot.value as! [String: AnyObject]
            print("GOT VALID \(postJSON)")
            let post = IdeaPost(message: postJSON["message"] as! String, byUser: postJSON["user"] as! String, withId: i.description)
            post.upvotes = postJSON["upvotes"] as! Int
            self.ideaPostDataSource.append(post)
            self.loadNext(i + 1)
        } else {
            // doesn't exhist
            print("GOT NULL RETURNING AT \(i)")
            self.doneLoading = true
            self.hitNull = true
            return
        }
    }
}

This recursive function essentially runs getting the value for key number i from firebase. If it is NSNULL it knows that is the last possible post to load and never does again. If NSNULL doesn't get hit but i % 29 == 0 then it returns as a base case so only 30 posts are loaded at a time (0 indexed). When I set doneLoading to true tableView.reloadData() is called using a property observer.

Here is a sample of what the array I am fetching looks like

"ideaPosts" : [ {
    "id" : 0,
    "message" : "Test",
    "upvotes" : 1,
    "user" : "Anonymous"
  }, {
    "id" : 1,
    "message" : "Test2",
    "upvotes" : 1,
    "user" : "Anonymous"
  } ]

1 comments

@Frank van Puffelen 2016-03-11 05:18:53

Update: we now also cover this question in an AskFirebase episode.

Loading many items from Firebase doesn't have to be slow, since you can pipeline the requests. But your code is making this impossible, which indeed will lead to suboptimal performance.

In your code, you request an item from the server, wait for that item to return and then load the next one. In a simplified sequence diagram that looks like:

Your app                     Firebase 
                             Database

        -- request item 1 -->
                               S  L
                               e  o
                               r  a
                               v  d
                               e  i
        <-  return item  1 --  r  n
                                  g
        -- request item 2 -->
                               S  L
                               e  o
                               r  a
                               v  d
                               e  i
                               r  n
        <-  return item  2 --     g
        -- request item 3 -->
                 .
                 .
                 .
        -- request item 30-->
                               S  L
                               e  o
                               r  a
                               v  d
                               e  i
                               r  n
                                  g
        <-  return item 30 --

In this scenario you're waiting for 30 times your roundtrip time + 30 times the time it takes to load the data from disk. If (for the sake of simplicity) we say that roundtrips take 1 second and loading an item from disk also takes one second that least to 30 * (1 + 1) = 60 seconds.

In Firebase applications you'll get much better performance if you send all the requests (or at least a reasonable number of them) in one go:

Your app                     Firebase 
                             Database

        -- request item 1 -->
        -- request item 2 -->  S  L
        -- request item 3 -->  e  o
                 .             r  a
                 .             v  d
                 .             e  i
        -- request item 30-->  r  n
                                  g
        <-  return item  1 --     
        <-  return item  2 --      
        <-  return item  3 --
                 .
                 .
                 .
        <-  return item 30 --

If we again assume a 1 second roundtrip and 1 second of loading, you're waiting for 30*1 + 1 = 31 seconds.

So: all requests go through the same connection. Given that, the only difference between get(1), get(2), get(3) and getAll([1,2,3]) is some overhead for the frames.

I set up a jsbin to demonstrate the behavior. The data model is very simple, but it shows off the difference.

function loadVideosSequential(videoIds) {
  if (videoIds.length > 0) {
    db.child('videos').child(videoIds[0]).once('value', snapshot => {
      if (videoIds.length > 1) {
        loadVideosSequential(videoIds.splice(1), callback)
      }
    });
  }
}

function loadVideosParallel(videoIds) {
  Promise.all(
    videoIds.map(id => db.child('videos').child(id).once('value'))
  );
}

For comparison: sequentially loading 64 items takes 3.8 seconds on my system, while loading them pipelined (as the Firebase client does natively) it takes 600ms. The exact numbers will depend on your connection (latency and bandwidth), but the pipelined version should always be significantly faster.

@Kato 2016-03-11 22:53:56

Nice, Puf! Also, chaining promises (jQuery.whenAll(), q.all(), or Promise.all()) can be very handy here if you need all items loaded, but still want to grab them in parallel, before taking some action.

@Frank van Puffelen 2016-03-11 23:16:31

Cool. Didn't even think of that, even though I have been using it. :-)

@Shirane85 2017-02-13 23:34:42

Am i missing something? You are calling to loadVideosSequential only after the previous "once" fetched the snapshot, similar to the how @Big_Mac_24 does but with different API call. Can you explain the difference please?

@Perry 2017-04-11 04:56:57

@FrankvanPuffelen You are right from the performance point of view but what if one of these calls didn't return due to any type of error? How can you 'cancel' rest of the pending requests if anyone out of these is failed. In case of sequential requests, we can get to know in code that which request failed. Please share your thoughts. Thanks.

@pejalo 2017-05-02 06:47:10

"The Promise.all() method [...] rejects with the reason of the first promise that rejects."

@DarkNeuron 2017-05-25 16:35:37

Perry, you wanna use Observables if you wanna cancel the rest.

@Muhammad chhota 2017-09-23 05:57:15

How we can do Promise.all in android? How we can load all the data in android

@Konstantin Konopko 2017-12-05 14:47:11

Please describe this case for Android

@Laurent Maquet 2018-02-08 11:01:00

Good to know ! But how can we pipeline several request and retrieving data from several Database refs in Swift ? Is there an equivalent of Promise.all in swift ?

@Frank van Puffelen 2018-02-08 15:00:31

I see some promising top results here: google.com/…

@Muhammad Adil 2018-03-13 12:58:59

@KonstantinKonopko Android SDK has been updated and there you can get whole datasnapshot against your query, So better you add GeoFire keys inside your original objects.

@OscarVGG 2018-06-26 12:38:56

How can I accomplish this in iOS/swift? Since the iOS sdk is callback driven, the observe functions don’t return promises or signals that I could map

Related Questions

Sponsored Content

2 Answered Questions

[SOLVED] Firebase's snapshot.value cast to Bool suddenly returning nil

1 Answered Questions

[SOLVED] Put users name as NavBar Item

4 Answered Questions

[SOLVED] Swift Error - Use of undeclared type 'cell' - Collection View

1 Answered Questions

[SOLVED] Observe single event within observer single event

2 Answered Questions

1 Answered Questions

0 Answered Questions

Enabling Firebase persistence data returns weird result

1 Answered Questions

Sponsored Content