Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No reference doc how to use Kafka Adapter to stream the data to kafka #1800

Open
pauldintel opened this issue Jan 30, 2024 · 3 comments
Open

Comments

@pauldintel
Copy link

I have looked at user guide and couldn't locate how to use Adapter Kafka : https://docs.nosqlbench.io/introduction/showcase/
I have streaming usecase to generate sample data and stream o kafka broker via producer .
Any help would be appreciated .
thanks

@jshook
Copy link
Contributor

jshook commented Jan 31, 2024

This is an oversight in the published docs. At the time the last site refresh happened, the kafka adapter was disabled for dependency refactoring.
You can find the missing docs here:
kafka in-tree resources

@pauldintel
Copy link
Author

pauldintel commented Feb 1, 2024

thank @jshook . Sorry being newbies here do you mean it is not disabled now and okay to use ?
I tried to launch this using nb5 :

nb5 run driver=kafka -vv cycles=100 threads=2 num_clnt=2 yaml=kafka_producer.yaml config=kafka_config.properties bootstrap_server=PLAINTEXT://localhost:9092

hit by this error :

6858 DEBUG [scenarios:001] METRICS      gauge added: loop3_kB_read
7073 DEBUG [scenarios:001] SCENARIO     -> invoking main scenario script (interpreted from logs/nQxXzt4.js)
7629 WARN  [scenarios:001] ActivityDef  Auto replacing deprecated activity param 'yaml=kafka_producer.yaml' with new 'workload=kafka_producer.yaml'.
7629 INFO  [scenarios:001] PARAMS       putting param workload=kafka_producer.yaml
7636 INFO  [scenarios:001] PARAMS       setting param alias=kafka_producer.yaml
7640 DEBUG [scenarios:001] PARAMS       getting entry set for (4)/{driver=kafka, threads=2, workload=kafka_producer.yaml, alias=kafka_producer.yaml, cycles=100, num_clnt=2, config=kafka_config.properties, bootstrap_server=PLAINTEXT://localhost:9092}
7653 ERROR [scenarios:001] SCENARIO     Error in scenario, shutting down. ({})

javax.script.ScriptException: java.lang.RuntimeException: Unable to load path 'kafka_producer.yaml'
at io.nosqlbench.adapters.api.activityconfig.OpsLoader.lambda$loadPath$0(OpsLoader.java:58)
at java.base/java.util.Optional.orElseThrow(Optional.java:403)

@dave2wave
Copy link
Contributor

@pauldintel Is your kafka_producer.yaml in the same directory where you are launching nb5?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

3 participants