Add api root to taxii2 server

[gzerphISA]

I'm having trouble with the syntax in the data-configuration.yml file in order to implement an api-root on the system. I have the taxii2 server up and responding but cannot get an api-root to show in the system.

Please let me know what I'm doing wrong as I'm getting lost in the documentation.

Thank you,

data-config yml
`apiroots:

• id: test_api_root
  default: true
  title: Test API Root
  description: Main API Root
  is_public: true

collections:

• id: collection-a
  api_root_id: test_api_root
  title: Main Collection
  description: Main collection Information
  is_public: true
  is_public_write: true

accounts:

• username: test
  password: test
  permissions:
  collection-a: modify
• username: admin
  password: admin
  is_admin: yes`

opentaxii yml
`---

domain: "localhost:9000"

support_basic_auth: yes
return_server_error_details: no

auth_api:
class: opentaxii.auth.sqldb.SQLDatabaseAPI
parameters:
db_connection: sqlite:////tmp/auth.db
create_tables: yes
secret: SECRET-STRING-NEEDS-TO-BE-CHANGED
token_ttl_secs: 3600

taxii2:
allow_custom_properties: true
contact: devops@hq.example.com
description: TAXII2 Server
max_content_length: 2048
persistence_api:
class: opentaxii.persistence.sqldb.Taxii2SQLDatabaseAPI
parameters:
create_tables: true
db_connection: sqlite:////tmp/data.db
public_discovery: true
title: Taxii2.1 Service

logging:
opentaxii: info
root: info`

[scottpendlebury]

Hi, I have the same issue - did you manage to solve it?

[gzerphISA]

I did not. I got frustrated with the lack of documentation around Taxii2 and ended up writing my own server.

[scottpendlebury]

Ah, thanks for the response

[gzerphISA]

Sorry. Wish i had a better answer for you. I do plan on releasing this project as it gets developed. I'll let you know when thats done, but hopefully you are able to figure out a quick answer.

Best of luck

[scottpendlebury]

No need to apologise - yes, please do let me know when you release your project. It seems as though there are few options when it comes to TAXII servers so another implementation would be appreciated by many I'd expect

[andrewbeard]

From what I can tell there is no way to create TAXII2 api roots and collections from the data config file at the moment. There are some cli utilities that can do it behind the scenes, however (opentaxii-add-api-root and opentaxii-add-collection). I'm going to look into cleaning that up in the future but I'm trying to fix some of the base TAXII2 functionality first.

[dragsu]

I found https://docs.eclecticiq.com/ic/current/install-configure-upgrade/configure-ic-settings/configure-services/opentaxii/

It looks like you have to modify TAXII2_API_ROOT variable in platform_settings.py

[andrewbeard]

I found https://docs.eclecticiq.com/ic/current/install-configure-upgrade/configure-ic-settings/configure-services/opentaxii/

It looks like you have to modify TAXII2_API_ROOT variable in platform_settings.py

That seems to be for their product based on the opentaxii package, but this project doesn't actually contain or reference a platform_settings.py file. It and TAXII2_API_ROOT appear nowhere in the code. It sounds like they have something else built on top of it to create an API root that isn't included.

[dragsu]

@andrewbeard You are right. I started playing with opentaxii-add-api-root cli tool and getting,

Traceback (most recent call last):
  File "/usr/local/bin/opentaxii-add-api-root", line 8, in <module>
    sys.exit(add_api_root())
  File "/usr/local/lib/python3.8/site-packages/opentaxii/cli/persistence.py", line 110, in add_api_root
    app.taxii_server.servers.taxii2.persistence.api.add_api_root(
AttributeError: 'SQLDatabaseAPI' object has no attribute 'add_api_root'

[andrewbeard]

@dragsu I think you need to change your persistence api class from SQLDatabaseAPI to Taxii2SQLDatabaseAPI.

[dragsu]

Yes that helped. Thank you @andrewbeard .

Just posting here for info.
My custom opentaxii.yml

domain: "0.0.0.0:9000"
taxii1:
taxii2:
  persistence_api:
    class: opentaxii.persistence.sqldb.Taxii2SQLDatabaseAPI
    parameters:
      db_connection: mysql://<username>:<password>@<db-url>:<port>/<db-name>
      create_tables: yes
  title: "EIQ TAXII2 Server"
  max_content_length: 209715200
  public_discovery: true

auth_api:
  class: opentaxii.auth.sqldb.SQLDatabaseAPI
  parameters:
    db_connection: mysql://<username>:<password>@<db-url>:<port>/<db-name>
    create_tables: yes
    secret: secret

logging:
  opentaxii: info
  root: info

This should create the required tables for taxii2.

Then run the below commands to create api_root, collection, and account.

opentaxii-add-api-root -t <some title> # Create an API root
opentaxii-add-collection -h # should return available api roots
opentaxii-add-collection  -r <api root> -t <some title>
opentaxii-create-account -u <username> -p <password>

If all goes well, you should be able to test the server using the following curl commands.

curl -kg -u '<username>:<password>' -H 'Content-Type: application/taxii+json;version=2.1' -H 'Accept: application/taxii+json;version=2.1' http://<IP>:<PORT>/taxii2/ # Returns server info

curl -kg -u '<username>:<password>' -H 'Content-Type: application/taxii+json;version=2.1' -H 'Accept: application/taxii+json;version=2.1' http://<IP>:<PORT>/taxii2/<api-root>/ # Returns API root info

curl -kg -u '<username>:<password>' -H 'Content-Type: application/taxii+json;version=2.1' -H 'Accept: application/taxii+json;version=2.1' http://<IP>:<PORT>/taxii2/<api-root>/collections/<collection-id>/objects/ # Should return any objects in the given collection

[dragsu]

When I try to push data as below,

curl -kgX POST -u '<username>:<password>' -H 'Content-Type: application/taxii+json;version=2.1'  -H 'Accept: application/taxii+json;version=2.1' -d '{
  "more": false,
  "objects": [
    {
      "id": "marking-definition--613f2e26-407d-48c7-9eca-b8e91df99dc9",
      "type": "marking-definition",
      "spec_version": "marking-definition",
      "name": "TLP:WHITE",
      "created": "2017-01-20T00:00:00.000Z",
      "modified": "2022-07-29T13:42:44.472979Z",
      "definition": {
        "tlp": "white"
      },
      "definition_type": "tlp"
    }
  ]
}' http://<ip>:<port>/taxii2/bd433926-c4a2-4607-8bab-1878c04152c1/collections/6337f0ad-a618-47e5-8a02-419c49e2ff24/objects/

I get an error response with the following message.

<html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

[andrewbeard]

@dragsu I would recommend opening a new issue for that one, since it seems like you've gotten past the scope of the api root. When you open it please include the logs from the server and not just the client response.

[dragsu]

@andrewbeard I did a bit more debugging and found the reason for the error. Currently spec_version in opentaxii_stixobject is set to varchar(10) and in my payload, I got more than 10 chars for the spec_version. I altered the table to make it varchar(50) and that helped to solve the problem.

Now I can send the STIX objects to the server and store them successfully. However, there is a "TypeError('Object of type UUID is not JSON serializable')" error when the server is trying to return the response back to the client. Will debug that and post the solution if I manage to fix it.

Edit:
This required more tinkering with OpenTAXII code. This is due to trying to json dump UUID objects. I had to update the code in OpenTAXII/opentaxii/server.py and OpenTAXII/opentaxii/persistence/manager.py to pass the collection and job ids as strings instead of UUIDs. After the change, I managed to push the payload into the TAXII server and receive the response successfully.

Does anyone know whether owners accept bug fixes?

[andrewbeard]

@dragsu My understanding is that spec version should be something like "2.0" or "2.1". I know I saw an issue earlier with spec_version being a copy of type, though. In any event there should probably be a length check in the server rather than an obscure error message.

Unfortunately with the UUID JSON error you've found a VERY well known issue with the current code in the repo (#243). There are multiple PRs for that issue pending, but anyone with write access seems to have disappeared over a year ago. Aside from recommending a fork of the repo I can't help much there.

[dragsu]

That sounds like a good reason to move to Medallion for TAXII 2.x.