I’m trying to conceptualize how to organize using the DI tools in aurelia with a IndexDB for persistance.
My goals are:
- High performance persistence - we’re storing 20-50 thousand records. I messed with LocalForage but the performance is just way too slow.
- Fall through caching - where data can be accessed from the cache, and if not found, can be pulled from our upstream rest api.
My initial thought is:
- Use 1 database for my entire application, and 1 ObjectStore for each “type” of data. We have a bunch of different REST api’s that each serve their own database types.
- Develop a base “IndexDB” service that provides the initial database connection and the
upgrade (migrations) required for the database setup
- Develop another service “SimpleStore”, and Inject the IndexDB service into the simple store.
Does this structure make sense?
well with these loads, are you sure IndexDb is what you’re really looking for? Perhaps something like https://rxdb.info/ would suite you more?
Whatever Service/s you create, just make sure the storage internals do not leak to public methods as chances are you’ll need to shift things around.
Aside from that I dunno what type of answer you’d be looking for as your description sounds a bit vague. What is the problem you’re thinking of or trying to avoid?
The main issue I’m trying to conceptualize a workaround for is the async nature of the IndexDB initialization. There’s a bunch of setup that needs to happen before the service is ready and I’m really quite unsure of how to set that up. For example, would I put that in the constructor of the “IndexDB” service?
To answer your other question - I guess I’m not sure, I haven’t really done something like this before so I’m learning step by step about the options that exist. We already have a websocket service in place that is serving the loads well - I just want to implement a caching layer to help improve this.
ahh ok, thats clearer now. so what you could do is initialize your db and service in main.ts before you actually au.start() so that bootstrapping gets delayed. This way, once in the app, the connection is already in place and ready to roll.
I’d highly recommend validating your idea first before going all in. I remember IndexDB has/had serious perf issues with with large numbers of docs, plus you can never be sure of the max size you get as its different per browser.
Stiring 50k+ docs on the client, without knowing the use case, sounds a bit crazy 
Perhaps you can elaborate on what you’re trying to achieve?
Edit: here is an article that will perhaps help Slow IndexedDB · RxDB - Documentation
This is good to know - I might have to investigate alternative options indeed.
Basically we’re rendering geographic data on a mapping interface - the map performance is fine with 50K features, but getting the features downloaded takes a lot of time, and a lot of network traffic - and I was hoping to improve that using client side caching.
Maybe using one of those sqlite options would be more efficient for this given how poor performance is with idb. I have been working on a simple sample (not going on all on this yet). Just trying to vet out what would work best :). Thanks for your insight though - client side persistence is a larger challenge than I anticipated!
Edit: words
well wouldnt the client cache the route once he hits up your rest endpoint given http cache headers are available?
if you need more control a service worker with the cache api might work wonders (Service worker overview - Chrome Developers).
I understand that the DB was merely as storage option not a necessity and a cached JSON would have sufficed as well.
EDIT:
for convenience the workbox tooling might help as well Workbox | Google Developers
The data is fairly dynamic so header/http caching isn’t really realistic. Also, we’re using a real-time web socket back-end FeathersJS which websockets lose the capable of providing http caching.
I think you’ve provided some valuable feedback here and things to chew on though, I’m going to try experimenting a bit more with those alternatives to IndexDb that were linked in those articles.
1 Like