Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to set consumer option "auto.offset.reset" ? #342

Closed
nobusue opened this issue Mar 28, 2016 · 8 comments
Closed

How to set consumer option "auto.offset.reset" ? #342

nobusue opened this issue Mar 28, 2016 · 8 comments

Comments

@nobusue
Copy link

nobusue commented Mar 28, 2016

I'd like to consume from latest message when I don't have committed offset (ex: new consumer group).
At v0.3.2, HighLevelConsumer consumes from oldest message, but it isn't expected.

@haio
Copy link
Member

haio commented Mar 31, 2016

You can set fromOffset true and give the offset you wish to consume in payload, example:

consumer = new HighLevelConsumer(
        client,
        [
            { topic: 't', offset: 1000 }
        ],
        { fromOffset: true }
    );

@nobusue
Copy link
Author

nobusue commented Apr 1, 2016

Thanks haio, but it doesn't match my needs.
I just intend to start consume latest message for "fresh consumer" (= there is no committed offset entry in zookeeper). In other language like Java, Kafka client API supports to set option map, like "auto.offset.reset=latest".
Is there any way to set Kafka client API option map directly?

@haio
Copy link
Member

haio commented Apr 1, 2016

This feature is not supported at the moment.

@CharlesWall
Copy link
Contributor

I'll have a PR coming for picking up from the latest offset.

@ghost
Copy link

ghost commented Feb 9, 2017

@CharlesWall any option like auto.offset.reset=latest , sometime soon for 0.8.* version of kafka for node client.

@hyperlink
Copy link
Collaborator

@shubSK I wouldn't hold my breath waiting for that.

@CharlesWall
Copy link
Contributor

I haven't looked at any of this stuff since May. I learned how this module worked to write the fix I needed. @shubSK You should consider doing the same.

@hyperlink
Copy link
Collaborator

The solution is to use ConsumerGroup which supports both earliest and latest. I don't think it's worth updating the HLC since it has been deprecated by Kafka project.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants