Electron app Database with Dexie.js ( indexedDB ) and web worker

Database for a electron app made with Dexie.js, which is a indexedDB wrapper.

Overview

  • Create dexie database
  • Running database in a worker
  • Import and export database
  • Make it persistent

Electron app and Dependencies

git clone https://github.com/electron/electron-quick-start
cd electron-quick-start
npm install

2. install dexie.js package
It is compatible with all modern browser and have a good community size.

npm install dexie --save

3. install dexie-export-import package
This package extends ‘dexie’ module with new methods for importing / exporting databases to / from blobs. This require low memory uses and works in web workers (better speed + doesn’t lock GUI).

npm install dexie-export-import --save

4. install stream-to-blob package
This package converts a Readable Stream into a Blob

npm install stream-to-blob --save

5. install node-readlines package
Reading file line by line may seem like a trivial problem, but in node, there is no straightforward way to do it

npm install n-readlines --save

Enable node.js in electron app

webPreferences: {   preload: path.join(__dirname, 'preload.js'),
nodeIntegration: true,
nodeIntegrationInWorker: true
}
  • nodeIntegration enables node.js in renderer processes.
  • nodeIntegrationInWorker enables node.js in web workers.

Database initialization

  1. Create a databaseInit.js file and add below code in it
var Dexie = require('dexie');
var database = new Dexie("dexieDB");
database.version(1).stores({ Friends: 'name,quality,issue'})database.open().catch(function(error){ console.error("ERROR: "+ error);});module.exports = { database}
  • Here we are initializing a friend database with name, quality and issue filed.
  • Catch any error(if any) while opening database.
  • Export database reference so that other script can access database.

2. Add Sample data to database
Create file called data.txt and add below data in it:

ash<|>purely lovable<|>anger immatureray<|>trust worthy<|>angerkevin<|>understanding<|>have limitsrayon<|>innocence<|>idiotsaki<|>nature and understanding<|>too much of open
  • <|> is a divider/delimiter for data you can use whatever you want.

Database web worker script

  1. Create a databaseWorker.js file and add below code in it.
const {database} = require('./databaseInit.js');const lineByLine = require('n-readlines');var duplicates=0, addedRecords=0, splitter;const liner = new lineByLine('./data.txt');let line, lineNumber = 0;database.transaction('rw', database.Friends, async () => {  while (line = liner.next()) {      splitter = line.toString().split('<|>');      await database.Friends.add({      name: splitter[0],      quality: splitter[1],      issue: splitter[2]    }).then(function(){      addedRecords++      lineNumber++;    }).catch(function (e) {      console.log(e.message);      duplicates++;      lineNumber++;    });  }}).then(function(){  postMessage("Duplicates " + duplicates.toString() + " Added " + addedRecords.toString());//use when you are using a web worker}).catch(error => {  console.error(error);});
  • Import database references to add data in it.
  • Import n-readlines package to read file line by line, then initialize data file with line variable.
  • Start a database transaction(this will rollback in case of failure).
  • Start a loop for each line and split line using delimiter then add it in database.
  • If line is added successfully then increase addedRecords variables and if there is any failure then increase duplicates or failure case.
  • Once transaction is done send success and failure cases to renderer process (if this script is running as web worker ).

Call web worker

  1. Edit renderer.js and add below code in it
var worker = new Worker('./databaseWorker.js');

worker.onmessage = function(event){
console.log("Database worker process is ", event.data);
worker.terminate();

document.querySelector("h1").innerHTML = (event.data);

//console.log("worker is done working ");
};
worker.onerror = function (event){
console.error(event.message, event);
};
  • initialize web worker with databaseWorker.js script.
  • onmessage is used to catch any message sent from web worker.
  • onerror is used to catch any error occurred in web worker.

Execute app to check run database

npm start
  • you should see this:
  • you can look at database from View -> Toggle Developer Tools -> Applications -> IndexedDB -> dexieDB -> Friends

Import and Export database and Persist database

const {database} = require('./databaseInit.js');require('dexie-export-import');const toBlob = require('stream-to-blob');const fileSystem = require('fs');var export_database = async function export_database(){  console.log("Exporting Database");  const blob = await database.export({prettyJson: true});  const text = await new Response(blob).text();   try{    fileSystem.writeFile("ExportedDatabase.json", text, function(error){     if(error){        console.log(error);      }    });  }catch(error){    console.error(''+error);  }  console.log("Exported");};var import_database = async function import_database(){  console.log("Importing Database");  const stream = fileSystem.createReadStream("ExportedDatabase.json");  const blob = await toBlob(stream);  try{    await database.import(blob);  }catch(error){    console.log('IMPORT ERROR: '+ error );  }  console.log("Imported");};
async function try_persist_without_promting_user() { if (!navigator.storage || !navigator.storage.persisted) { return "never"; } let persisted = await navigator.storage.persisted(); if (persisted) { return "persisted"; } if (!navigator.permissions || !navigator.permissions.query) { return "prompt"; // It MAY be successful to prompt. Don't know. } const permission = await navigator.permissions.query({ name: "persistent-storage"});if (permission.status === "granted") { persisted = await navigator.storage.persist(); if (persisted) { return "persisted"; } else { throw new Error("Failed to persist"); } } if (permission.status === "prompt") { return "prompt"; } return "never";}var init_storage_persistence = async function init_storage_persistence() { console.log("persisting data"); const persist = await try_persist_without_promting_user(); switch (persist) { case "never": return "Not possible to persist storage"; case "persisted": return "Successfully persisted storage silently"; case "prompt": return "Not persisted, but we may prompt user when we want to."; }}module.exports = { export_database, import_database, init_storage_persistence}
  • Import databaseInit.js script to open database.
  • Import daxie-export-import , stream-to-blob , fs packages.
  • Export database: here we first create a blob from dexie database, then we convert blob into readable text(json), lastly write that text in a file.
  • Import database: read database file then convert it in a blob, lastly import blob into database
  • Both import export work in dexie database format of json.
  • Persist database: electron delete indexedDB database once it exceeds certain quota, to avoid that we persist database.
  • init_storage_persistence try to persist database print result on console.
  • Lastly export all functions to be accessed by other scripts.

Export and Persist database created in previous steps

  • Edit renderer.js to add below code in it
var importExportDB = require("./importExportDatabase");importExportDB.export_database();var isPersisted = importExportDB.init_storage_persistence();
isPersisted.then(function(value){
console.log(value);
});
  • first we import importExportDatabase.js
  • Export database
  • Try to persist database and return result on console

2. Run app to see result

npm start
  • you should see these results on console
  • Now check project directory and open ExportedDatabase.json

Project Code

Support my work

Additional Possibilities

  • dexie can be used to design more complex database so please checkout their documentation.
  • You can ask any question in comments or find me on twitter.

Full Stack Developer and Concept Designer (https://atiqgauri.github.io/)