Skip to content

Insert millions of documents into a mongoDB

November 11, 2022 | 11:22 PM

Background

I had read a lot about mongoDB and it’s performance when it comes to storing json objects. Therefor I decided to make a little lab setup and write a test program.

The Setup

Virtualbox with a VM 1 vCPU and 18GB of RAM. Clean Ubuntu 22.04 install.

Installation of mongoDB followed the official mongoDB guide for Ubuntu.

Decisions are made

I wanted something simpel and able to insert millions of documents. The program should be able to receive inputs so it could be used with various settings.

Furthermore and due to the nature of the documents in mongoDB, I needed to create a random data structure to some extend. This required a additional input to determine the string length of the randomized dataset.

In order to use the complete CPU power of my own host, I decided to make the program multithreaded.

As per my usual approach … I only care for function and solving my issue. So the code tends to be rather straight up and not devided into functions main etc.

The Program

See it and fiddle with it on Github : https://github.com/CRPrinzler/mongodb-multiprocessor-insert-test

Enjoy !