Skip to content

Persistence for REST resources in the filesystem or a redis database

License

Notifications You must be signed in to change notification settings

lbovet/vertx-rest-storage

Repository files navigation

vertx-rest-storage

Build Status Maven Central

Persistence for REST resources in the filesystem or a redis database.

Stores resources in a hierarchical way according to their URI. It actually implements a generic CRUD REST service.

It uses usual mime mapping to determine content type so you can also use it as a web server. Without extension, JSON is assumed.

The following methods are supported on leaves (documents):

  • GET: Returns the content of the resource.
  • PUT: Stores the request body in the resource.
  • DELETE: Deletes the resource.

The following methods are supported on intermediate nodes (collections):

  • GET: Returns the list of collection members. Serves JSON and HTML representations.
  • POST (StorageExpand): Returns the expanded content of the sub resources of the (collection) resource. The depth is limited to 1 level. See description below
  • DELETE: Delete the collection and all its members.

Runs either as a module or can be integrated into an existing application by instantiating the RestStorageHandler class directly.

Run it

  1. clone the repository.
  2. run gradle build -x test
  3. run the fatjar with `java -jar build/libs/rest-storage-x.x.x-all.jar
  4. you get a rest-storage, that stores to the filesystem in the directory where you started it. If you want to use the rest-storage with redis, you have to pass the configuration over a json file with -conf conf.json

Features

GET

Invoking GET request on a leave (document) returns the content of the resource.

GET /storage/resources/resource_1

Invoking GET request on a collection returns a list of collection members.

GET /storage/resources/

Parameters

Parameter Description
limit defines the amount of returned resources
offset defines the amount of resources to skip. Can be used in combination with limit to provide pageing functionality
Examples

Given a collection of ten items (res1-res10) under the path /server/tests/offset/resources/

Request Returned items
GET /server/tests/offset/resources/?limit=10 all
GET /server/tests/offset/resources/?limit=99 all
GET /server/tests/offset/resources/?limit=5 res1,res10,res2,res3,res4
GET /server/tests/offset/resources/?offset=2 res2,res3,res4,res5,res6,res7,res8,res9
GET /server/tests/offset/resources/?offset=11 no items (empty array)
GET /server/tests/offset/resources/?offset=2&limit=-1 res2,res3,res4,res5,res6,res7,res8,res9
GET /server/tests/offset/resources/?offset=0&limit=3 res1,res10,res2
GET /server/tests/offset/resources/?offset=1&limit=10 res10,res2,res3,res4,res5,res6,res7,res8,res9

The returned json response look like this:

{
  "resources": [
    "res1",
    "res10",
    "res2",
    "res3",
    "res4"
  ]
}

StorageExpand

The StorageExpand feature expands the hierarchical resources and returns them as a single concatenated json resource.

Having the following resources in the storage

key: data:test:collection:resource1     value: {"myProp1": "myVal1"}
key: data:test:collection:resource2     value: {"myProp2": "myVal2"}
key: data:test:collection:resource3     value: {"myProp3": "myVal3"}

would lead to this result

{
    "collection" : {
        "resource1" : {
            "myProp1": "myVal1"
        },
        "resource2" : {
            "myProp2": "myVal2"
        },
        "resource3" : {
            "myProp3": "myVal3"
        }
    }
}
Usage

To use the StorageExpand feature you have to make a POST request to the desired collection to expand having the url paramter storageExpand=true. Also you wil have to send the names of the subresources in the body of the request. Using the example above, the request would look like this:

POST /yourStorageURL/collection with the body:

{
    "subResources" : ["resource1", "resource2", "resource3"]
}

Lock Mechanism

The lock mechanism allows you to lock a resource for a specified time. This way only the owner of the lock is able to write or delete the given resource. To lock a resource, you have to add the following headers to your PUT / DELETE request.

Headers Type Default value Description
x-lock String The owner of the lock.
x-lock-mode silent silent Any PUT or DELETE performed on this resource without the valid owner will have no effect and get 200 OK back.
reject Any PUT or DELETE performed on this resource without the valid owner will have no effect and get 409 Conflict back.
x-lock-expire-after long 300 Defines the lock lifetime. The default value is set to 300 seconds.
x-expire-after long Defines the lifetime of a resource

Warning: The lock will be always removed if you perform a DELETE on a collection containing a locked resource. There is no check for locks in collections.

Store data compressed

In order to optimize the memory usage when using the redis storage, it's possible to store resources compressed using the gzip compression algorithm.

To store a resource compressed, add the following header to the PUT request:

x-stored-compressed: true

When making a GET request to a compressed resource, the resource will be uncompressed before returning. No additional header is required!

Restrictions

The data compression feature is not compatible with all vertx-rest-storage features. The following listing contains the restrictions of this feature:

  • Data compression is available in redis storage only
  • Data compression cannot be used with merge=true url parameter concurrently. Such PUT requests will be rejected.
  • Compressed resources cannot be used in storageExpand requests. storageExpand requests to a collection containing a compressed resource will be rejected.
  • If a resource is already stored in a different compression state (state = not compressed, compressed) as the compression of sent resource, the stored resource will be overwritten in every case. Like this we prevent unexpected behaviour considering the etag mechanism.

Configuration

The following configuration values are available:

Property Type Default value Description
root common . The prefix for the directory or redis key
storageType common filesystem The storage implementation to use. Choose between filesystem or redis
port common 8989 The port the mod listens to.
prefix common / The part of the URL path before this handler (aka "context path" in JEE terminology)
storageAddress common resource-storage The eventbus address the mod listens to.
editorConfig common Additional configuration values for the editor
redisHost redis localhost The host where redis is running on
redisPort redis 6379 The port where redis is running on
expirablePrefix redis rest-storage:expirable The prefix for expirable data redis keys
resourcesPrefix redis rest-storage:resources The prefix for resources redis keys
collectionsPrefix redis rest-storage:collections The prefix for collections redis keys
deltaResourcesPrefix redis delta:resources The prefix for delta resources redis keys
deltaEtagsPrefix redis delta:etags The prefix for delta etags redis keys
lockPrefix redis rest-storage:locks The prefix for lock redis keys
resourceCleanupAmount redis 100000 The maximum amount of resources to clean in a single cleanup run

Configuration util

The configurations have to be passed as JsonObject to the module. For a simplyfied configuration the ModuleConfigurationBuilder can be used.

Example:

ModuleConfiguration config = with()
		.redisHost("anotherhost")
		.redisPort(1234)
		.editorConfig(new JsonObject().put("myKey", "myValue"))
		.build();

JsonObject json = config.asJsonObject();

Properties not overriden will not be changed. Thus remaining default.

To use default values only, the ModuleConfiguration constructor without parameters can be used:

JsonObject json  = new ModuleConfiguration().asJsonObject();

Storage types

Currently there are two storage types supported. File system storage and redis storage.

File System Storage

The data is stored hierarchically on the file system. This is the default storage type when not overriden in the configuration.

Redis Storage

The data is stored in a redis database. Caution: The redis storage implementation does not currently support streaming. Avoid transfering too big payloads since they will be entirely copied in memory.

Dependencies

This module uses Vert.x v3.2.0 (or later), so Java 8 is required.

Use gradle with alternative repositories

As standard the default maven repositories are set. You can overwrite these repositories by setting these properties (-Pproperty=value):

  • repository this is the repository where resources are fetched
  • uploadRepository the repository used in uploadArchives for releases
  • snapshotRepository the repository used in uploadArchives for snapshot
  • repoUsername the username for uploading archives
  • repoPassword the password for uploading archives

Bitdeli Badge

About

Persistence for REST resources in the filesystem or a redis database

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •