diff --git a/README.md b/README.md index c247fd701..9df6243b2 100644 --- a/README.md +++ b/README.md @@ -2,8 +2,6 @@ The official MongoDB connector for the LoopBack framework. -Please see the full documentation at [loopback.io](http://loopback.io/doc/en/lb3/MongoDB-connector.html). - ## Installation In your application root directory, enter this command to install the connector: @@ -18,9 +16,28 @@ If you create a MongoDB data source using the data source generator as described ## Creating a MongoDB data source +For LoopBack 4 users, use the LB4 [Command-line interface](https://loopback.io/doc/en/lb4/Command-line-interface.html) to generate a DataSource with MongoDB connector to your LB4 application. Run [`lb4 datasource`](https://loopback.io/doc/en/lb4/DataSource-generator.html), it will prompt for configurations such as host, post, etc. that are required to connect to a MongoDB database. + +After setting it up, the configuration can be found under `src/datasources/.datasource.ts`, which would look like this: + +```ts +const config = { + name: 'db', + connector: 'mongodb', + url: '', + host: 'localhost', + port: 27017, + user: '', + password: '', + database: 'testdb', +}; +``` + +
For LoopBack 3 users + Use the [Data source generator](http://loopback.io/doc/en/lb3/Data-source-generator.html) to add a MongoDB data source to your application. The generator will prompt for the database server hostname, port, and other settings -required to connect to a MongoDB database. It will also run the `npm install` command above for you. +required to connect to a MongoDB database. It will also run the `npm install` command above for you. The entry in the application's `/server/datasources.json` will look like this: @@ -40,6 +57,8 @@ The entry in the application's `/server/datasources.json` will look like this: Edit `datasources.json` to add any other additional properties that you require. +
+ If your username or password contains special characters like `@`, `$` etc, encode the whole username or password using [encodeURIComponent](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/encodeURIComponent). @@ -47,16 +66,17 @@ Eg: `pa$$wd` would become `pa%24%24wd`. ### Connection properties -| Property | Type   | Description | -| --- | --- | --- | -| connector | String | Connector name, either `"loopback-connector-mongodb"` or `"mongodb"`. | -| database | String | Database name | -| host | String | Database host name | -| password | String | Password to connect to database | -| port | Number | Database TCP port | -| url | String | Connection URL of form `mongodb://user:password@host/db`. Overrides other connection settings (see below). | -| user | String | Username to connect to database | -| authSource | String | Authentification database name (optional). Usually `"admin"` value. | +| Property | Type   | Description | +| ---------- | ---------------- | ---------------------------------------------------------------------------------------------------------- | +| connector | String | Connector name, either `"loopback-connector-mongodb"` or `"mongodb"`. | +| database | String | Database name | +| host | String | Database host name | +| name | String | Name of the datasource in the app | +| password | String | Password to connect to database | +| port | Number | Database TCP port | +| url | String | Connection URL of form `mongodb://user:password@host/db`. Overrides other connection settings (see below). | +| user | String | Username to connect to database | +| authSource | String | Optional. Authentification database name. Usually `"admin"` value. | If you run a MongoDB with authentification ([Docker's example here](https://github.com/docker-library/docs/tree/master/mongo#mongo_initdb_root_username-mongo_initdb_root_password)), you need to specify which database to authenticate against. More details can be found in [MongoDB documentation on Authentification Methods](https://docs.mongodb.com/manual/core/authentication/#authentication-methods). The default value is usually `"admin"`, like in the official docker image. @@ -64,24 +84,56 @@ If you run a MongoDB with authentification ([Docker's example here](https://gith ### Additional properties -- **allowExtendedOperators**: Set to `true` to enable using MongoDB operators such as -`$currentDate`, `$inc`, `$max`, `$min`, `$mul`, `$rename`, `$setOnInsert`, `$set`, `$unset`, `$addToSet`, -`$pop`, `$pullAll`, `$pull`, `$pushAll`, `$push`, and `$bit`. Default is `false`. -- **enableGeoIndexing**: Set to `true` to enable 2dsphere indexing for model properties -of type `GeoPoint`. This allows for indexed ```near``` queries. Default is `false`. -- **lazyConnect**: - - Default is `false`. - - If set to `true`, the database instance will not be attached to the datasource and the connection is deferred. - - It will try to establish the connection automatically once users hit the endpoint. If the mongodb server is offline, the app will start, however, the endpoints will not work. -- **disableDefaultSort**: Set to `true` to disable the default sorting - behavior on `id` column, this will help performance using indexed columns available in mongodb. -- **collation**: Specify language-specific rules for string comparison, such as rules for lettercase and accent marks. See [`MongdoDB documentation`](https://docs.mongodb.com/manual/reference/collation/) for details. It can also be used to create [`case insensitive indexes`](https://docs.mongodb.com/manual/core/index-case-insensitive/). - -### Setting the url property in datasource.json - -You can set the `url` property to a connection URL in `datasources.json` to override individual connection parameters such as `host`, `user`, and `password`. - -Additionally, you can override the global `url` property in environment-specific data source configuration files, for example for production in `datasources.production.json`, and use the individual connection parameters `host`, `user`, `password`, and `port`. To do this, you _must_ set `url` to `false`, null, or “” (empty string). + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
PropertyTypeDefaultDescription
allowExtendedOperatorsBooleanfalseSet to true to enable using MongoDB operators such as $currentDate, $inc, $max, $min, $mul, $rename, $setOnInsert, $set, $unset, $addToSet, $pop, $pullAll, $pull, $pushAll, $push, and $bit.
enableGeoIndexingBooleanfalseSet to true to enable 2d sphere indexing for model properties of type GeoPoint. This allows for indexed near queries.
lazyConnectBooleanfalseWhen set to true, the database instance will not be attached to the datasource and the connection is deferred. It will try to establish the connection automatically once users hit the endpoint. If the MongoDB server is offline, the app will start, however, the endpoints will not work.
disableDefaultSortBooleanfalseSet to true to disable the default sorting behavior on id column, this will help performance using indexed columns available in MongoDB.
collationStringN/ASpecify language-specific rules for string comparison, such as rules for letter-case and accent marks. See MongoDB documentation for details. It can also be used to create case insensitive indexes.
+ +### Setting the url property in datasource.ts + +You can set the `url` property to a connection URL in `.datasources.ts` to override individual connection parameters such as `host`, `user`, and `password`. E.g `loopback:pa55w0rd@localhost:27017/testdb`. + +
For LoopBack 3 users + +For LB3 users, you can override the global `url` property in environment-specific data source configuration files, for example for production in `datasources.production.json`, and use the individual connection parameters `host`, `user`, `password`, and `port`. To do this, you _must_ set `url` to `false`, null, or “” (empty string). If you set `url` to `undefined` or remove the `url` property altogether, the override will not work. For example, for production, use `datasources.production.json` as follows (for example) to overide the `url` setting in `datasources.json: @@ -101,18 +153,36 @@ For example, for production, use `datasources.production.json` as follows (for e For more information on setting data source configurations for different environments, see [Environment-specific configuration](https://loopback.io/doc/en/lb3/Environment-specific-configuration.html#data-source-configuration). +
+ ### Using the mongodb+srv protocol + MongoDB supports a protocol called `mongodb+srv` for connecting to replica sets without having to give the hostname of every server in the replica set. To use `mongodb+srv` as the protocol set the `protocol` connection property in the datasource.json to `mongodb+srv`. For example: +```ts +const config = { + name: 'db', + connector: 'mongodb', + host: 'myserver', + database: 'testdb', + protocol: 'mongodb+srv', +}; +``` + +
For LoopBack 3 users + ```javascript "mydb": { + "connector": "mongodb", "host": "myserver", "database": "test", - "protocol": "mongodb+srv", - "connector": "mongodb" + "protocol": "mongodb+srv" } ``` + +
+ Note: the port is not specified when using the `mongodb+srv` protocol and will be ignored if given. ## Security Considerations @@ -122,13 +192,11 @@ MongoDB Driver allows the `$where` operator to pass in JavaScript to execute on To protect users against this potential vulnerability, LoopBack will automatically **remove** the `$where` and `mapReduce` operators from a query before it's passed to the MongoDB Driver. If you need to use these properties from within LoopBack programmatically, you can disable the sanitization by passing in an `options` object with `disableSanitization` property set to `true`. **Example:** -```js -Post.find( - {where: {$where: 'function() { /*JS function here*/}'}}, - {disableSanitization: true}, - (err, p) => { - // code to handle results / error. - } + +```ts +await PostRepository.find( + { where: { $where: "function() { /*JS function here*/}" } }, + { disableSanitization: true } ); ``` @@ -138,7 +206,9 @@ See [LoopBack types](http://loopback.io/doc/en/lb3/LoopBack-types.html) for de ### LoopBack to MongoDB types -Type conversion is mainly handled by Mongodb. See ['node-mongodb-native'](http://mongodb.github.io/node-mongodb-native/) for details. +Type conversion is mainly handled by MongoDB. See ['node-mongodb-native'](http://mongodb.github.io/node-mongodb-native/) for details. + +
For LoopBack 3 users ## Customizing MongoDB configuration for tests/examples @@ -177,27 +247,40 @@ The .loopbackrc file is in JSON format, for example: **Note**: user/password is only required if the MongoDB server has authentication enabled. `"authSource"` should be used if you cannot log in to your database using your credentials. +
+ ## Running tests ### Own instance + If you have a local or remote MongoDB instance and would like to use that to run the test suite, use the following command: + - Linux + ```bash MONGODB_HOST= MONGODB_PORT= MONGODB_DATABASE= CI=true npm test ``` + - Windows + ```bash SET MONGODB_HOST= SET MONGODB_PORT= SET MONGODB_DATABASE= SET CI=true npm test ``` ### Docker + If you do not have a local MongoDB instance, you can also run the test suite with very minimal requirements. + - Assuming you have [Docker](https://docs.docker.com/engine/installation/) installed, run the following script which would spawn a MongoDB instance on your local: + ```bash source setup.sh ``` + where ``, `` and `` are optional parameters. The default values are `localhost`, `27017` and `testdb` respectively. + - Run the test: + ```bash npm test ``` @@ -227,16 +310,54 @@ make benchmarks The results will be output in `./benchmarks/results.md`. -## strictObjectIDCoercion flag +## Handling ObjectId + +MongoDB uses `ObjectId` for its primary key, which is an object instead of a +string. In queries, string values must be cast to `ObjectId`, otherwise they are +not considered as the same value. Therefore, you might want to specify the data +type of properties to enforce `ObjectId` coercion. Such coercion would make sure +the property value converts from ObjectId-like string to `ObjectId` when it +accesses to the database and converts `ObjectId` to ObjectId-like string when +the app gets back the value. (An ObjectId-like string is a string that has length 12 or 24 and has the format of an `ObjectId` i.e /^[0-9a-fA-F]{24}\$/.) + +LoopBack provides two scopes to handle such coercion: per model or per property. Please check the following to see which configuration meets your requirements. + +{% include important.html content="please make sure you are using `loopback-connector-mongodb` package version 5.2.1 +or above to handle `ObjectId` properly." %} + +- No `ObjectId` coercion: CRUD operations can be operated with non-ObjectId-like + string or ObjectId-like string ids. + +- Enforce `ObjectId` coercion: the property value can only be `ObjectId` or + ObjectId-like string, otherwise it will be rejected. + +Enforcing `ObjectId` coercion can be done by setting the flag +`strictObjectIDCoercion` in the **model definition** or by specifying +`dataType: ObjecId` in the **property definition**. + +### Model scope + +This scope would do the conversion for all properties in the model. -In version 1.17.0, the id of string type is being converted to ObjectID, when the string length is 12 or 24 and has the format of an ObjectID i.e /^[0-9a-fA-F]{24}$/. -To avoid this issue, the `strictObjectIDCoercion` flag should be set to true in the model-definition file. It is also possible to enable this flag on a per method bases by passing it in as part of the options object. +```ts +@model({settings: { + strictObjectIDCoercion: true +}}) +export class User extends Entity { +@property({ + type: 'string', + id: true, + }) + id: string; +... +} +``` -### model-definition.js +
For LoopBack 3 users ```js { - "name": "myModelName", + "name": "User", "base": "PersistedModel", "idInjection": false, "options": { @@ -246,60 +367,109 @@ To avoid this issue, the `strictObjectIDCoercion` flag should be set to true in ... } ``` -boot-script.js + +
+ +### Property scope + +This scope would only convert an ObjectId-like string to `ObjectId` with a certain property in the model. + +```ts +@property({ + type: 'string', + id: true, + mongodb: {dataType: 'ObjectId'} + } + id: string; +``` + +
For LoopBack 3 users ```js -'use strict'; -var util = require('util'); - -module.exports = function(app) { - var db = app.dataSources.mongoDs; - var myModelName = app.models.myModelName; - - db.automigrate(function(err) { - if (err) throw err; - console.log('Automigrate complete'); - - myModelName.create([{ - id: '59460487e9532ae90c324b59', - name: 'Bob', - }, { - id: '59460487e9532ae90c324b5a', - name: 'Sam', - }, { - id: '420', - name: 'Foo', - age: 1, - }, { - id: '21', - name: 'Bar', - }], function(err, result) { - if (err) throw err; - console.log('\nCreated instances of myModelName: ' + util.inspect(result, 4)); - - myModelName.find({where: {id: {inq: ['59460487e9532ae90c324b59', - '59460487e9532ae90c324b5a']}}}, - function(err, result) { - if (err) throw err; - console.log('\nFound instance with inq: ' + util.inspect(result, 4)); - }); - }); - }); -}; + "properties": { + { + "id": { + "type": "String", + "id": true, + "required":true, + "mongodb": {"dataType":"ObjectId"} + }, + // .. + } + } +``` + +
+ +Also notice that for RELATIONS, if the primary key/source key has set to enforce ObjectId coercion +(no matter by `strictObjectIDCoercion: true` or `dataType: 'ObjectId'`). The corresponding foreign key will need to have it +set as well to make sure relations work smoothly. + +```ts +@model() +export class User extends Entity { +// source key +@property({ + type: 'string', + id: true, + mongodb: {dataType: 'ObjectId'} + }) + id: string; +... +} + +@model(// ) +export class Address extends Entity { + ... + // foreign key + @belongsTo(() => User, + {}, //relation metadata goes in here + {// property definition goes in here + mongodb: {dataType: 'ObjectId'} + }) + UserId: string; +} ``` -### Customize collection/field names +## Customize collection/field names -`loopback-connector-mongodb` allows you to have different collection and field names from the models. For example, the following setting would define a collection with custom name `Custom_Collection`, and it has a custom field name `Custom_Content` in the database: +`loopback-connector-mongodb` allows you to have different collection and field names from the models. Such configurations can be added to the model definition and the property definition respectively as `mongodb:{ : }`. For example, the following setting would define a collection with custom name `Custom_Collection_User`, and it has a custom field name `Custom_Name` in the database: -{% include code-caption.html content="/common/models/model.json" %} +{% include code-caption.html content="/src/models/User.model.ts" %} + +```ts +@model({ + settings: { + // model definition goes in here + mongodb: { collection: "Custom_Collection_User" }, + }, +}) +export class User extends Entity { + @property({ + type: "string", + id: true, + generated: true, + }) + id: string; + + @property({ + type: "string", + mongodb: { + fieldName: "Custom_Name", + }, + }) + name?: string; +} +``` + +
For LoopBack 3 users ```js { - "name": "MyModel", + "name": "User", "options": { "mongodb": { - "collection": "Custom_Collection", //custom name + "collection": "Custom_Collection_User", //custom name }, }, "properties": { @@ -309,48 +479,18 @@ module.exports = function(app) { "id": true, "required":true, }, - "content": { + "name": { "type": "String", - "mongodb": {"fieldName": "Custom_Content"},}, //custom name + "mongodb": {"fieldName": "Custom_Name"},}, //custom name } }, } ``` -{% include warning.html content=" -Custom field names can only be applied to **non-id properties**. Customizing the id property would cause errors. Only the default name `_id` is allowed in the DB. -" %} - -### Per method basis - -```js -myModelName.find( - {where: {id: {inq: ['59460487e9532ae90c324b59', '59460487e9532ae90c324b5a']}}}, - {strictObjectIDCoercion: true}, - function(err, result) { - // ... - } -) -``` - -## dataType: 'ObjectID' - -You can set a model property's `mongodb` property definition `dataType` to "ObjectID" to enforce ObjectID coercion -irrespective of the `strictObjectIDCoercion` setting. +
-In the following example, the `id` and `xid` will be coerced to `ObjectID` even if `strictObjectIDCoercion` is set to true. - -```js -const User = ds.createModel( - 'user', - { - id: {type: String, id: true, mongodb: {dataType: 'ObjectID'}}, - xid: {type: String, mongodb: {dataType: 'ObjectID'}} - }, - {strictObjectIDCoercion: true} -); -``` +{% include important.html content="Since in MongoDB `_id` is reserved for the primary key, LoopBack **does not** allow customization of the field name for the id property. Please use `id` as is. Customizing the id property would cause errors." %} ## Release notes - * 1.1.7 - Do not return MongoDB-specific _id to client API, except if specifically specified in the model definition +- 1.1.7 - Do not return MongoDB-specific `_id` to client API, except if specifically specified in the model definition