By est


2011-12-05 10:38:19 8 Comments

I am making a analytics system, the API call would provide a Unique User ID, but it's not in sequence and too sparse.

I need to give each Unique User ID an auto increment id to mark a analytics datapoint in a bitarray/bitset. So the first user encounters would corresponding to the first bit of the bitarray, second user would be the second bit in the bitarray, etc.

So is there a solid and fast way to generate incremental Unique User IDs in MongoDB?

5 comments

@MarcoSantana 2019-11-08 13:24:35

The best way I found to make this to my purpose was to increment from the max value you have in the field and for that, I used the following syntax:

maxObj = db.CollectionName.aggregate([
  {
    $group : { _id: '$item', maxValue: { $max: '$fieldName' } } 
  }
];
fieldNextValue = maxObj.maxValue + 1;

$fieldName is the name of your field, but without the $ sign.

CollectionName is the name of your collection.

The reason I am not using count() is that the produced value could meet an existing value.

The creation of an enforcing unique index could make it safer:

db.CollectionName.createIndex( { "fieldName": 1 }, { unique: true } )

@Konstantin Pribluda 2011-12-05 10:47:55

You can, but you should not https://web.archive.org/web/20151009224806/http://docs.mongodb.org/manual/tutorial/create-an-auto-incrementing-field/

Each object in mongo already has an id, and they are sortable in insertion order. What is wrong with getting collection of user objects, iterating over it and use this as incremented ID? Er go for kind of map-reduce job entirely

@est 2011-12-05 10:57:18

The problem is concurrenty. iterate would emit duplicate incremental ID.

@Konstantin Pribluda 2011-12-05 11:49:50

You need this ID only for analysis, not data storage. IIUC, you need sequential ID only as index for your array, so you can build bitset. You can achieve it without storing this incremental ID in database, and you can build your bit-array without retrieving data to client

@est 2011-12-05 13:22:50

It's quite unwise to get incr ID by iteration everytime, especially you are dealing with millions of users per datapoint. Dong a MAU would require like 30x iterations.

@mnemosyn 2011-12-05 15:12:48

It's unwise to use incremental sequences when you have millions of users in the first place. However, millions of users doesn't exactly play well with bit arrays either, does it? I find it hard to tell what exactly you're trying to achieve. Concurrency will not be a problem using findAndModify. Also see mongodb.org/display/DOCS/Object+IDs and the HiLo Algorithm: stackoverflow.com/questions/282099/whats-the-hi-lo-algorithm

@est 2011-12-12 03:23:59

I just want to store some Redis bitmap data in Mongo blog.getspool.com/2011/11/29/… for later queries.

@Konstantin Pribluda 2011-12-12 06:49:14

It seems that you need mapping between some UID and bit in a redis bitset. If that is the case, you can also store this mapping in redis pretty effectively. And it would scale well, because mapping is write-one and read only

@est 2011-12-12 09:59:29

Redis has RAM limitations, and not ideal for old data persistancy. MongoDB can handle much better. I really need to generate Unique Auto Incremental ID in MongoDB.

@Negarrak 2018-09-27 21:18:02

The thing is that I am too lazy to type more than a couple of cipher for an ID; and Mongo creates huge ids since the beginning.

@shadow0359 2019-06-05 12:34:29

As mentioned in the answer, mongo objectId's are not strictly sortable in insertion order. stackoverflow.com/questions/31057827/…

@Darshan Vithani 2019-01-03 08:12:10

First Record should be add

"_id" = 1    in your db

$database = "demo";
$collections ="democollaction";
echo getnextid($database,$collections);

function getnextid($database,$collections){

     $m = new MongoClient();
    $db = $m->selectDB($database);
    $cursor = $collection->find()->sort(array("_id" => -1))->limit(1);
    $array = iterator_to_array($cursor);

    foreach($array as $value){



        return $value["_id"] + 1;

    }
 }

@Alex Nicholas 2017-04-29 12:33:35

I know this is an old question, but I shall post my answer for posterity...

It depends on the system that you are building and the particular business rules in place.

I am building a moderate to large scale CRM in MongoDb, C# (Backend API), and Angular (Frontend web app) and found ObjectId utterly terrible for use in Angular Routing for selecting particular entities. Same with API Controller routing.

The suggestion above worked perfectly for my project.

db.contacts.insert({
 "id":db.contacts.find().Count()+1,
 "name":"John Doe",
 "emails":[
    "[email protected]",
    "[email protected]"
 ],
 "phone":"555111322",
 "status":"Active"
});

The reason it is perfect for my case, but not all cases is that as the above comment states, if you delete 3 records from the collection, you will get collisions.

My business rules state that due to our in house SLA's, we are not allowed to delete correspondence data or clients records for longer than the potential lifespan of the application I'm writing, and therefor, I simply mark records with an enum "Status" which is either "Active" or "Deleted". You can delete something from the UI, and it will say "Contact has been deleted" but all the application has done is change the status of the contact to "Deleted" and when the app calls the respository for a list of contacts, I filter out deleted records before pushing the data to the client app.

Therefore, db.collection.find().count() + 1 is a perfect solution for me...

It won't work for everyone, but if you will not be deleting data, it works fine.

@ckinfos 2017-09-17 07:37:29

is there any special reason that the mongodb states that you have to use a function and a counters sequence, instead of your soultion as db.xxx.find.count+1? does maybe transcation processing messing things? do your solution works well in web server environment cruds' ? thank you for your answer

@zephos2014 2018-01-10 21:47:30

This would not be good in a concurrent setup. You could easily get documents with the same _id if they did the count at the same time.

@Alex Nicholas 2018-03-20 23:12:49

absolutely! In my instance, I am not having to deal with concurrency or sharding at all so I don't have any problems using a find().Count()+1 As in my original answer, this won't work for everyone in every situation, but it has definitely has worked in my particular scenario. App has been in production now for nearly 12 months with no issues regarding my incrementing id's.

@ofir_aghai 2019-10-25 06:26:10

This solution is bad, because you changed the history! In one time can be a document with id 3 and another time it can happen again. But there is not any relation between the documents that has this id's

@expert 2013-01-21 06:11:15

As selected answer says you can use findAndModify to generate sequential IDs.

But I strongly disagree with opinion that you should not do that. It all depends on your business needs. Having 12-byte ID may be very resource consuming and cause significant scalability issues in future.

I have detailed answer here.

@Tino Costa 'El Nino' 2017-12-25 10:29:44

You can if you want to, I disagree too, because that's a mongo inbuilt feature to .createIndex ( { "number" : 1 }, { unique : true } ) where the one represents increment and -1 otherwise

Related Questions

Sponsored Content

10 Answered Questions

[SOLVED] NoSQL (MongoDB) vs Lucene (or Solr) as your database

10 Answered Questions

[SOLVED] Reset auto increment counter in postgres

8 Answered Questions

[SOLVED] How to set initial value and auto increment in MySQL?

3 Answered Questions

[SOLVED] Insert a document while auto incrementing a sequence field in MongoDB

  • 2018-03-26 21:03:35
  • Rotem Varon
  • 2006 View
  • 1 Score
  • 3 Answer
  • Tags:   c# mongodb

2 Answered Questions

[SOLVED] How to create an auto increment java variable

2 Answered Questions

2 Answered Questions

[SOLVED] How can I create an auto increment field on meteor?

1 Answered Questions

[SOLVED] Simulating an Oracle sequence with MongoDB

  • 2015-05-27 14:58:07
  • Spiff
  • 155 View
  • 2 Score
  • 1 Answer
  • Tags:   mongodb

2 Answered Questions

[SOLVED] Auto-increment a field with leading 0's in MongoDB schema

1 Answered Questions

[SOLVED] Unique ids in MongoDB

  • 2013-02-12 20:11:25
  • Niklas9
  • 102 View
  • 0 Score
  • 1 Answer
  • Tags:   mongodb

Sponsored Content