Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple minor bug fixes #20

Closed
wants to merge 2 commits into from
Closed

Multiple minor bug fixes #20

wants to merge 2 commits into from

Conversation

mahendra
Copy link
Collaborator

  • When you initiate a producer with a non-existant queue, the queue is
    created. However this partition info is not reflected in KafkaClient()
    immediately. So, we wait for a second and try loading it again.

    Without this fix, if we do producer.send_messages() after creating a new
    queue, the library will throw a StopIteration exception.

  • In SimpleConsumer(), the defaults are not as mentioned in the comments. Fix this (or do we change the documentation?)

  • There was a problem with the way the consumer iterator worked.
    for eg: assume that there were 10 messages in the queue/topic
    and you iterate over it as -

    for msg in consumer:
         print (msg)
    

    At the end of this, offset that is saved is 10.
    So, if you run the above loop again, the last message (10) is repeated.

    This can be fixed by adjusting the offset counter before fetching the message

  • Avoid some code repeat in consumer.commit()

  • Fix a bug in send_offset_commit_request() invocation in consumer.py

  • Fix missing imports

* When you initiate a producer with a non-existant queue, the queue is
  created. However this partition info is not reflected in KafkaClient()
  immediately. So, we wait for a second and try loading it again.

  Without this fix, if we do producer.send_messages() after creating a new
  queue, the library will throw a StopIteration exception.

* In SimpleConsumer(), the defaults are not as mentioned in the comments.
  Fix this (or do we change the documentation?)

* There was a problem with the way the consumer iterator worked.
  for eg: assume that there were 10 messages in the queue/topic
  and you iterate over it as -

  for msg in consumer:
       print (msg)

  At the end of this, 'offset' that is saved is 10.
  So, if you run the above loop again, the last message (10) is repeated.

  This can be fixed by adjusting the offset counter before fetching
  the message

* Avoid some code repeat in consumer.commit()

* Fix a bug in send_offset_commit_request() invocation in consumer.py

* Fix missing imports
@mahendra
Copy link
Collaborator Author

@mumrah I wonder if this will cause conflicts. Let me know if it happens.

Conflicts:
	kafka/consumer.py
@mahendra
Copy link
Collaborator Author

@mumrah I have fixed the conflicts and pushed it.

@mumrah
Copy link
Collaborator

mumrah commented May 29, 2013

Thanks, @mahendra I actually have this merged locally with some changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants