javascript - Node.js, store object in memory or database? - Stack Overflow

Am developing a node js application, which reads a json list from a centralised db

List Object is around 1.2mb(if kept in txt file)

Requirement is like, data is to be refreshed every 24 hours, so i kept a cron job for it

Now after fetching data i keep it into a db(couchbase) which is locally running on my server

Data access is very frequent, i get around 1 or 2 req per sec and nearly all req need that Object

Is it good to keep that Object as a in memory object in node js or to keep it in local db ?

What are the advantages and disadvantages of both ?

Object only read for all requests , only written once by cron job

it's a high end system, i7 quad core, 16gb ram

Am developing a node js application, which reads a json list from a centralised db

List Object is around 1.2mb(if kept in txt file)

Requirement is like, data is to be refreshed every 24 hours, so i kept a cron job for it

Now after fetching data i keep it into a db(couchbase) which is locally running on my server

Data access is very frequent, i get around 1 or 2 req per sec and nearly all req need that Object

Is it good to keep that Object as a in memory object in node js or to keep it in local db ?

What are the advantages and disadvantages of both ?

Object only read for all requests , only written once by cron job

it's a high end system, i7 quad core, 16gb ram

• javascript
• node.js
• memory-management
• database-performance

Share Improve this question Follow edited Aug 4, 2016 at 8:27 Aishwat Singh asked Aug 4, 2016 at 8:17 Aishwat SinghAishwat Singh 4,47933 gold badges2929 silver badges4949 bronze badges 6

• A "InProc" object is going to be substantially faster to access than any DB. But obviously uses that amount of memory. You also need to be careful about memory leaks. That all said, the proper anwer to this is totally depdant on your wider implementation so your likely the only one that can answer this. Try it and see – Liam Commented Aug 4, 2016 at 8:21
• Depends, if you start copying the object you could consume huge amounts of RAM, but if done correctly an in memory object would be a lot faster performance wise – Paradoxis Commented Aug 4, 2016 at 8:22
• while trying i just kept it as a json object and in module.exports i exposed that object , so that i can access that from other locations . Also while accessing that from other locations i pre-check if its null ? is it good ? – Aishwat Singh Commented Aug 4, 2016 at 8:25
• That does sound like the way to go. Still, is the data mutable? – Randy Commented Aug 4, 2016 at 8:28
• If you need performance keep it in memory. If you need performance but can't afford the RAM you will end up buying more RAM in the end. So it depends on if you need the performance. If you don't need performance you don't need to keep it in memory - just fetch it from the db. Consider that languages that can't keep data in memory use a separate server to do it: memcached. So it's a good idea if you need performance. Personally at 1 or 2 req per second I would say you don't need it. Node.js can fortably handle 500+ requests per second. I'd do it once you get to around 100 req/sec – slebetman Commented Aug 4, 2016 at 8:32

 |  Show 1 more ment

2 Answers 2

Sorted by: Reset to default 5

1. It depends from your hardware
2. If this object is immutable, per requests, it's better to keep it in memory. If no - depends.
3. In any case workflow open connection to db - fetch data - return result - free data will consume more resources than caching in memory.

For example, in our project we processing high definition images, and keep all objects in memory - 3-7mb in raw format. Tests shows that this is much efficient than usage of any caching systems, such as redis or couch base.

I would keep the most recent version as memory object, and store it as well. That way you have a backup if anything crashes. If you edit the file however, I would only keep it as database object.

Accessing the DB for that object every 2 seconds would probably work fine, but 1.2MB of memory is not that much and if you can keep that contained, your server won't likely run into problems.

The DB is a little slow pared to memory, but has the advantage to (most likely) be thread-safe. If you would edit the document, you could run into thread problems with a memory object.

You know the application and the requirements, you should be able to tell if you would need a thread-safe database, or if you need to safe your memory on the server. If you don't know, we need to see the actual code and use-cases to tell you what you could do best.