-
-
Notifications
You must be signed in to change notification settings - Fork 3.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How to share the connection pool across multiple databases #1124
Comments
It's not presently supported. On Sep 25, 2012, at 7:05 PM, Rex Huang notifications@github.com wrote:
|
+1 |
If a node.js server needs to connect to 20+ database in a same server, and using a node.js cluster with 4 process, then the connections be will easily over 500 (the default PoolSize is 5). This scenario is not rare for a multi-tenancy architecture. Could somebody kindly provide a review of my tweak? The tweak reuses connections of the same server (ReplicaSet not implemented yet). https://github.com/baoshan/mongoose/commit/e192bc94cb19f4c5d7b9e9cb52343bf6e82a178e |
@baoshan yeah we should expose this somehow. i'm not convinced yet that caching it as in your commit is the right approach, but i haven't looked into what it would take yet. |
+1 I'm looking for a way to have models over different databases (multi tenancy application). The source of Mongoose.prototype.model suggests that passing a connection is possible when compiling a new Model. But this feature is undocumented and i have feeling i will run into some issues when using this in production :-) Update: I've switched to Don Park's approach https://github.com/donpark/mongeese |
+1 I have tried nearly everything I could think of - including using mongeese... No luck... The biggest problem being, I need to talk to the correct database, based on the settings in my wrapper class and hold default schemas + the altered versions. @Reggino is that what you are doing with mongeese? I can't seem to make it work. |
O.k... I have just gotten my case to work - though I haven't checked for the amount of connections... So in case anybody else needs this, this is what works for me now (in it's most basic form): var mongoose = require('mongoose'); var db2 = mongoose.createConnection('mongodb://127.0.0.1/test2'); sites1.find(…)… |
@morphar Yes, that is what Mongeese does. Where mongoose is designed as a Singleton instance by default, mongeese is a 'hack' to be able to have multiple instances. Advantage over your solution is that you may use a second mongoose-object with it's own default connection. |
yes, the mongoose module is itself an instance of mongoose.Mongoose. creating more instances is definitely an option but not really what this ticket is about. this ticket is for reusing the existing connection pool. |
(Feeling totally newbie here...) |
I still think it would be nice to be able to share the connection pool - but I can imagine that would be harder in the case I wrote above. |
working on a solution for 3.6. |
will be in rc1 |
after further review we're pushing this to after 3.6.0. too big of a change this late. |
new require('mongoose').Mongoose now contains all constructors that require('mongoose') exposes. see https://github.com/donpark/mongeese relates to #1124
I don't want to stress things, but is there an ETA for this feature? I really need to know this for future plans on our multitenant application.... Thanks |
No ETA but I'd like it in v3.7. Last I researched it the driver behavior That said, the best way to get this implemented in a timely fashion is On Monday, May 6, 2013, Tim de Koning wrote:
Aaron |
closed. #1601 |
I would just like to add a comment regarding multi tenancy and mongoose. Seems to me that mongoose is not well designed for multi tenant applications since the models are hard coupled to the database. The API is designed for single databases, therefore you can do something like this: |
I don’t think mongoose is not suitable for a multi tenancy deployment. At least, no body stop us from architecting our application into a On Aug 13, 2014, at 4:13 PM, Manuel Astudillo notifications@github.com wrote:
|
@baoshan it may not be impossible, but if you aim to make a modular application I believe it is not suitable :). |
TL;DR @manast when I say From a data-governance point-of-view, one client (tenant) should have a separate database. Multiple clients' databases all share the same logic (that's what a service means, right?), so they share the same schemes (but not the same models), the same middlewares, and the same connection pools. |
@baoshan I agree in mostly everything you say. But I think it that what you would also need is to be able to share the Model, not model instances, but the Model class itself. Otherwise it does not scale. For instance I don't think it is efficient to create the model from a schema every time a user accesses your service... sure you could cache all the models in every user session, but seems to be overcomplicated and unneeded. |
In my scenario (imagine a REST-full data-service) every tenant (imagine an API key) has it's own schemas (and thus Models). As part of the service, a tenant can configure the schema/model himself. There is shared logic however in how e.g. validation is done and some business logic is handled. Shared "Models" (or even shared schemas) may be useful in some scenarios but it doesn't fit the bill everywhere. So i guess that's just a matter of how your app is organised. |
@manast I think we're agree with each user/tenant hold their own separate database, are we? And I believe each user's database are isolated totally, which means mostly query only access a single database, am I right? Otherwise we're not talking about multi-tenancy, we're talking about multiple users of a single tenant's application. The current problem is you do not want to instantiate a set of new models when the user(tenant) use your application, am I right? You can cache the models into a models pool. which will share a same connection pool, so it's resource / memory efficient. So you do not need to instaitiate new models (that may be time consuming). Do you think this is an acceptable solution? What's your suggestion if you feel that's not optimal? |
@baoshan I think we agree on the same things. Regarding my proposal, it is simple, just decouple the Model from the database. So when you need to access a Model in a given database you just do something like: What do you think? |
@manast I think I get your idea. According to your proposal:
Have I missed something? |
|
I agree with you that caching Currently, I have a simple middleware (express middleware here) which return the cached models according to the subdomain (the subdomain denotes the tenant). Then, for the 99 percents of my code, I have a set of models which is bounded to the requesting tenant. For my projects, one app server only serves tens of tenants, and the solution fits my current requirement and doesn't compromise my developing experience. I guess your application server needs to serves hundreds of thousands of tenants, am I right? Then you have to some test I think. |
@baoshan ah, ok. They just changed the api but it is still done with createConnection. |
@manast, Have you found any way to use mongoose for your use case. In our case we have hundreds of tenants that all use the same schema. If I understand from the above, the proposed solution is to create thousands of models then cache them myself somehow? |
@memelet Yes, we have been running with this solution for a few months now. we have like max 10 tenants so there has not been any performance issue. Basically what you need to do is this: var db = mongoose.createConnection(uri);
db.Schema = mongoose.Schema; And then access every model with: var User = db.models.User;
etc So it requires a new mongoose connection, unless mongoose multiplexes the connection it will result in several connections to mongodb, but still, having a few hundreds should be feasible, connections are not that expensive anyways. |
@manast Can you please provide a little detailed example or if you have found any better approach now? |
In node-mongodb-native, I can share the connection pool across multiple databases:
var mongodb = require("mongodb"), mongoserver = new mongodb.Server(host, port, server_options), db_connector = new mongodb.Db(name, mongoserver, db_options);
db_connector.open(callback);
And then I use
db_connector.db(name)
, this returns a new db instance that shares the connections off the previous instance but will send all commands to the database name . This allows for better control of resource usage in a multiple database scenario.HOW CAN I DO THIS IN MONGOOSE?
The text was updated successfully, but these errors were encountered: