« Back to home

Optimizing your App with EmberData (Part 1)

Posted on

Welcome!

Over the next few weeks we're going to build and optimize an application using EmberData. The application we're building is called Listicle. It's a deceptively simple app with many commonly seen data structures, and designed to incorporate some of the worst performance scenarios applications routinely encounter.

We're building Listicle as a starting point. We're going to start with an app with abysmal performance, and iterate until we achieve great performance. This series is for beginners and experts alike, regardless of whether you build applications with Ember, or if you even like EmberData—and especially if not.

EmberData Crash Course

If you've never used EmberData, I'd recommend learning a bit about the architecture and history from my presentation at EmberFest 2019. You can also read through the Guides.

Our Application

Listicle is an application full of Top 20 lists for you to read, with the simple setup of a feed to scroll through available lists. As the name suggests, each list has 20 items. And of course our lists are created by our wonderful staff of authors.

Let's look at how we model Author, List and Item.

// models/author.js
import Model, { attr, hasMany } from '@ember-data/model';

export default class Author extends Model {
  @attr name;
  @attr profileImageUrl;
  @hasMany('list', { async: true, inverse: 'author' })
  lists;
}

// models/list.js
import Model, { attr, hasMany } from '@ember-data/model';

export default class List extends Model {
  @attr title;
  @hasMany('item', { async: true, inverse: 'list' })
  items;
  @belongsTo('author', { async: true, inverse: 'lists' })
  author;
}

//models/item.js
import Model, { attr, belongsTo, hasMany } from '@ember-data/model';

export default class Item extends Model {
  @attr description;
  @belongsTo('list', { async: true, inverse: 'items' })
  list;
  @hasMany('facet', { async: true, polymorphic: true, inverse: 'item' })
  facets;
}

You'll notice each item also has facets (the reasons why this item is great!). In Listicle, every item included on a list has 5 characteristics that make it so great!

Because our Top 20 lists could be anything, facets are polymorphic: each item has its own type of facets with their own unique properties.

// models/facet.js
import Model, { attr, belongsTo } from '@ember-data/model';

// The base class for our polymorphic Facets
export default class Facet extends Model {
  @attr fieldName; // fieldName specifies the key of a polymorphic attr
  @belongsTo('item', { async: true, inverse: 'facets' })
  item;
}

// models/item1-facet.js
import { attr } from '@ember-data/model';
import Facet from './facet';

// All of our facets follow this convention
/*
export default Item<N>Facet extends Facet {
  @attr [fieldName];
}
*/
// So if the value of `fieldName` were color
export default Item1Facet extends Facet {
  @attr color;
}
Note: Yes, this Facet could be modeled without polymorphism by having a generic value attr. But this form of polymorphism is fairly common, and more importantly, "real" applications often have dozens to thousands of models. This setup gives us a convenient way of exploring data at both large and small scales.

One final note on our initial design. We're going to create an Adapter and a Serializer for every single Model type. This is to mirror a common mistake many Ember applications make when using EmberData.

Listicle is conceived as a small app shell full of rich content lists. But what happens as our content grows? On January 1st, 2020 Listicle opens for business and begins publishing one new post a day.

By mid-July Listicle has produced 200 lists! Our site, which early on felt snappy and fast to the users who poured in to read our hottest articles, now takes seemingly forever to load.  And where initially as engineers we were happy and productive, now our build times have slowed to a crawl.

What happened?

Initially (ignoring authors) there was only one list to fetch, with 20 items, and 5 facets. 26 total records, 4 total Model classes, 12 total classes, including Adapters and Serializers. But as Listicle has grown, we added more Models, Adapters, and Serializers to handle all these facets.

And now, 200 days in, we've got 200 lists, 4k items, 20k facets and 12009 total classes. Oof.

This may sound contrived, but this is far smaller yet still representative of what an application I write infrastructure for looked like 3 years ago. This helps to show why certain architectural choices for EmberData failed, leading too many apps to have bad performance by default. More importantly, refactoring this application will illuminate why other architectural choices in EmberData are solid, providing value to both large and small apps, and why we are rebuilding over the top of them.

Let's get started

Now that we've introduced Listicle, let's take a peek at the initial implementation. You can follow along with the code for this series by watching the listicle repository.

Most of the posts in this series won't skip over as much as this next bit will, since it's the optimizations that are interesting here and not the initial implementation itself. But before we dig in I want to give us a good sense of where we are starting from.

Building the app, time-travel edition

Our first step, generate the application we are building using Octane and Yarn.

ember new listicle --yarn -b @ember/octane-app-blueprint --no-welcome

Then we add our basic app and we use codegen to add our 12009 classes to the app (as well as fixture data to fetch for them).

One detail of the basic app implementation to not gloss over is that in our initial implementation we are loading each list individually. This is often how initial designs start, don't worry, it is not where we will end up.

Now, for some numbers

As part of our codegen commit above we added a generate script to package.json, running it will place the Adapters, Models, Serializers, and Fixtures into their proper directories. Now that we've done this, let's take a few measurements to have a baseline for where our build and render performance are starting from.

We will be making regular performance and asset-size checks on both our build and our app throughout this series and we will introduce the infrastructure we will use for that in one of our next posts.

time node --max-old-space-size=8192 ./node_modules/.bin/ember b -e production --output-path="./dists/initial-app"

Above, we are using time to get an accurate length for how long this build is going to take. We are also manually invoking the ember command with node so that we can allocate more memory for node to use as otherwise (hint) this build will be so big it will fail. Finally, we are saving the built assets into a special dists directory which we will check into source control.

Throughout this series, we will commit the built assets from various check-points so that we can compare them quickly later both with tooling and manually.

If you clone the listicle repo and run yarn install, you can serve these assets to do your own performance exploration by running the following command and then navigating to localhost:4200 in your web browser of choice.

ember s --path="./dists/initial-app"

Here's a screenshot of our gorgeous app.

Screenshot of Listicle
Screenshot of Listicle

After several minutes, our build completes, let's take a look.

Screenshot of Build Output
Screenshot of Build Output

Some stats:

  • slow: Our build took over 5.5 minutes
  • scream: Our app.js file is 20.74 Mb but compresses to 183.5 Kb
  • Our vendor.js file is 690 Kb but compresses to 174.9 Kb

Note I am not sure what compression algorithm is used or what settings it is used with above, later when we implement asset-size monitoring we will use Brotli with compression set to Max (11).

What does this translate to in app performance? I took a quick profile of a page load to see.

Screenshot of Chrome Dev Tools Timeline for app render
Screenshot of Chrome Dev Tools Timeline for app render

Some initial reactions are:

  • Eyeballing, we look to be spending 30% of our time or more compiling code
  • Normalization looks to be about 20% of our time
  • Each network response (200 of them) spends a good chunk of time getting processed afterwards.
  • There is a lot of data related costs being paid in the middle of our long-yellow-snail of rendering.

Feel free to pull down the repo and serve up the app to poke around a timeline and see if you can spot these things. They are areas we will be looking to optimize over the course of this series. I'd recommend putting your browser in incognito mode and disabling all extensions before doing any performance profiling. Almost every extension has some negative effect on performance, and my own extensions (a few ad blockers and the Ember inspector) decreased the performance here by nearly 50% for me.

One final measurement before we go.

Screenshot of Chrome Dev Tools Overview for app render
Screenshot of Chrome Dev Tools Overview for app render

Some notes:

  • This particular render took about 6 seconds. (There's also a few extra seconds from a delay with the profiler attaching)
  • Our JS Heap Size for memory is 500 Mb !!
  • We have 57 DOM Listeners !! (Considering we added none ourselves this is a bit of a surprise, We will fix this in a future commit)
  • We've got almost 340k DOM nodes !! (This series won't look into the benefits of occlusion culling but we will look at the effects pagination has here)

In a future post we will setup automated performance analysis using Tracerbench to get high-fidelity benchmarks with confidence. While timeline based exploration is critical for quickly spotting problem spots, it isn't a great tool for knowing if we've improved them or not except when the improvement is very large (and thus easier to observe).

In my next post, we'll look closer at the mock API produced by the codegen and explore the fixture data it returns.

In the mean time, you can learn more about the architecture of EmberData from my presentation at EmberFest 2019! If you have any questions feel free to ask them on this forum thread, on Twitter, or in the Ember Community Discord Server.