Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add the CHATHISTORY extension specification. #292

Closed
wants to merge 30 commits into from
Closed
Changes from 23 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
d572c78
Add scrollback batch type
Jan 2, 2017
ff83973
Update copyright year
Jan 2, 2017
25c7e8d
Add RPL_ISUPPORT 005 to command specification
Jan 2, 2017
577d9c5
Add draft/msgid support to scrollback command
Jan 3, 2017
e2d46f5
Require batch reply
Jan 3, 2017
3871fa5
Make draft/msgid optional
Jan 3, 2017
ff2ef80
Reimplement extension using chathistory batch type
Jan 3, 2017
8ae7bf9
Update URLs
Jan 3, 2017
d261046
Rename scrollback to chathistory for consistency
Jan 3, 2017
c4883c0
Add max_message_count parameter to ISUPPORT
Jan 3, 2017
382968a
Add error replies and draft/label
Jan 4, 2017
ed14f4c
Specify label as draft
Jan 4, 2017
78b39f1
Improve error reporting and remove default size
Jan 10, 2017
78208a6
Make draft/label recommended
Jan 10, 2017
f923267
Specify optional draft/label
Jan 10, 2017
3ae7331
Specify work-in-progress draft implementation
Feb 13, 2017
0af7450
Allow for positive and negative directional message counts
Mar 7, 2017
77b4a6a
Typo fix
Mar 7, 2017
45b3e0a
Make draft/labeled-response recommended
Mar 8, 2017
e5f366b
Specificy draft/label and draft/msgid purposes
Mar 8, 2017
545f645
Specify draft/label and draft/msgid purposes
Mar 8, 2017
89b5299
Merge branch 'master' of https://github.com/MuffinMedic/ircv3-specifi…
Mar 8, 2017
23cb023
Merge branch 'master' of https://github.com/ircv3/ircv3-specifications
Apr 3, 2017
9cc586a
Standardize errors
Aug 23, 2017
d054b8b
Remove `draft/` from `CHATHISTORY` command
Aug 23, 2017
a210bfc
Remove WIP draft prefix requirement
Aug 23, 2017
4042cec
Change error codes to less-resemble existing numerics
Aug 25, 2017
8759f86
e and z
Aug 25, 2017
02aba74
n
Aug 25, 2017
88a3e1b
Merge branch 'master' of https://github.com/ircv3/ircv3-specifications
Nov 4, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 88 additions & 0 deletions extensions/chathistory.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
---
title: IRCv3 chathistory extension
layout: spec
work-in-progress: true
copyrights:
-
name: "Evan Magaliff"
period: "2017"
email: "evan@muffinmedic.net"
---
## Notes for implementing work-in-progress version
This is a work-in-progress specification.

Software implementing this work-in-progress specification MUST NOT use the unprefixed `CHATHISTORY` command. Instead, implementations SHOULD use the `draft/CHATHISTORY` command to be interoperable with other software implementing a compatible work-in-progress version.

The final version of the specification will use an unprefixed command.

## Description
This document describes the format of the `chathistory` extension. The `chathistory` extension uses the [chathistory][batch/chathistory] batch type, adding a client-side command for requesting `chathistory` content from the server.

The `chathistory` extension adds an optional `draft/msgid` to the `chathistory` batch reply.

Clients MUST support the [`batch`][batch] and [`server-time`][server-time] capabilities. Clients SHOULD support the [`draft/labeled-response`][draft/labeled-response] and [`draft/msgid`][draft/msgid] capabilities.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By support, does that mean the client should have already negotiated batch and server-time before sending the command?

Should the CHATHISTORY command produce some error numeric when these capabilities are not negotiated?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

draft/msgid is a message tag and not a capability. To use draft/msgid message tag, the capability draft/message-tags should be enabled. (http://ircv3.net/specs/extensions/message-ids.html#dependencies)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding is that CAPS are negotiated before the ISUPPORT is sent - if the CAPs are not supported (as required by MUST), then CHATHISTORY should not appear in the 005. This is already standard behavior to my best of my knowledge.

Will update capability URL.


When a client with the above mentioned capabilities requests `chathistory` content from the server (using the `CHATHISTORY` command outlined below), the server should return to the client a single batch containing a number of desired raw IRC lines equal to the `message_count` parameter specified, ending directly before the given timestamp for a negative `message_count`, after the number of messages specified for a positive `message_count`, or with the message directly proceeding the one with the specified `draft/msgid`. The raw IRC lines should be formatted and returned to the client as they were originally, with the addition of the above capability tags.

The `server-time` should be the time at which the message was received from the IRC server and the `batch id` a unique ID for the entire batch. `draft/label` SHOULD be included and MUST be a unique ID used to identify the `chathistory` request and any replies. A `draft/msgid` to identify each individual message MUST be the `draft/msgid` included when each message was first received from the IRC server.

### `CHATHISTORY` Command
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't the spec say draft/CHATHISTORY command and use that in further examples? Since this spec does keep draft/msgid and not msgid in all examples.

Chathistory content can be requested by the client to the server by sending the `CHATHISTORY` command to the server. A `batch` must be returned by the server. If no content exists to return, an empty batch should be returned to avoid the client waiting for a reply. Command support is sent to the client as the RPL_ISUPPORT 005 numeric `:irc.host 005 nick CHATHISTORY=max_message_count :are supported by this server`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is max_message_count required? Does omitting a value and having CHATHISTORY in ISUPPORT mean there are no limits enforced?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's required. As per #292 (comment), it will be updating such that 0 can be used to indicate no limit exists.


Both the `message_count` and `max_message_count` MUST be non-zero integers and `max_message_count` MUST be positive. The client should not request a `message_count` with an absolute value greater than the `max_message_count` parameter sent in the command. If the `message_count` absolute value exceeds the `max_message_count`, server should return a number of lines equal to the `max_message_count` and the appropriate warning as described below.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SHOULD should be in caps like MUST (RFC2119)


#### Format
The `chathistory` content can requested using timestamps:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this might lead to some timing problems down the line, since the timestamp is not unique per message. A server may receive two messages at the very same time and only one of these make it to a connected client before it disconnects.

Then the client will reconnect and ask for messages since the last message it seen which would not include the missing message.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is required to account for certain use cases in which draft/msgid tags are not available, such as in ZNC 1.6+ logs, in the format of YYYY-MM-DD.log [HH:mm:ss].


@draft/label=ID CHATHISTORY target timestamp=YYYY-MM-DDThh:mm:ss.sssZ message_count
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

YYYY-MM-DDThh:mm:ss.sssZ implies a specific format and not any valid ISO8601 date. I think the spec should clearly specify what is supported (ISO 8601) instead of only mentioning this format. ISO8601 valid dates may be in other formats like 2017-04-21T12:45:50+00:00.

Copy link
Author

@ghost ghost Apr 21, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The format given consistent with the server-time extension.


Alternatively, content can be requested using a `draft/msgid`:

@draft/label=ID CHATHISTORY target draft/msgid=ID message_count
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we make this format match that in other IRC specifications for consistency.

@draft/label=ID CHATHISTORY <target> draft/msgid=<message_id> <message_count>

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also not seeing much clarification on what target can be. Is it a direct target like #channel. Does it support comma separated values? #channel,#other-channel? Can we use * as wildcard?

Copy link
Author

@ghost ghost Apr 21, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will specify the target as a single channel or query. The use of wildcards or multiple channels becomes problematic with draft/msgid, as each ID is unique only to a single message and the server would not know what to retrieve for each individual target.

What would the standard format be for the timestamp=YYYY-MM-DDThh:mm:ss.sssZ option? The use of server-time for either seems inappropriate.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will specify the target as a single channel or query.

Okay if there is no wildcard, I am not sure how a client would know about the unread queries they have opened for individual nicks. Is there anyway a client would be able to load the available chat history targets?

znc-playback currently allows the wildcard so that clients can ask for all queries during launch so that they can load the nick queries.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CHATHISTORY is not intended to be a replacement for a playback buffer. Unless the spec is reworked, it currently has no concept of "read messages".


If `message_count` is positive, content MUST be retrieved from after and not including the specified timestamp or `draft/msgid`. If the `message_count` is negative, content MUST be retrieved from before the specified timestamp or `draft/msgid`.

#### Errors and Warnings
If the server receives an improperly formatted `CHATHISTORY` command, the `CMD_INVALID` error code should be returned.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is CMD_INVALID I don't see mention of it here or in RFC1459/RFC2812.

Wouldn't ERR_NEEDMOREPARAMS be sent if the required parameters are missing and not "CMD_INVALID"? Or ERR_NOSUCHNICK/ERR_NOSUCHCHANNEL if the target is non-existent?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If target can be multiple targets, is it possible to receive the ERR_TOOMANYTARGETS numeric?


If the absolute value of `message_count` exceeds the `max_message_count`, warn code `MAX_MSG_COUNT_EXCEEDED` should be returned. The command should continue to be processed as described above.

If no `chathistory` exists to return, the server should return the appropriate error code. `ACCESS_DENIED` should be sent if the user requests content they do not have permission to view.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is ACCESS_DENIED defined?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wil get rid of it due to other privacy concerns.


### Examples
The examples below are written with `draft/msgid` and `draft/label` tags included. These tags are recommended.

#### Begin
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does each line need a title?

@draft/label=ID :irc.host BATCH +ID chathistory target
#### PRIVMSG
@batch=ID;draft/msgid=ID;time=YYYY-MM-DDThh:mm:ss.sssZ :nick!ident@host PRIVMSG target :message
#### NOTICE
@batch=ID;draft/msgid=ID;time=YYYY-MM-DDThh:mm:ss.sssZ :nick!ident@host NOTICE target :message
#### End
:irc.host BATCH -ID
#### Error
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should use a separate example for errors/warnings imo.

@draft/label=ID :nick!ident@host CHATHISTORY ERR ERROR_CODE
#### Warning
@draft/label=ID :nick!ident@host CHATHISTORY WARN WARN_CODE

## Use Cases
The batch type and supporting command are useful for allowing an "infinite scroll" type capability within the client. A client will, upon scrolling to the top of the active window or a manual trigger, may request `chathistory` from the server and, after receiving returned content, append it to the top of the window. Users can repeat this historic scrolling to retrieve prior history until limitations are met (see below).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't CHATHISTORY command need to support requesting a range or messages before the given parameters. Currently it returns messages after a specified time or msgid. To implement this behaviour we would need to know the content before instead.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If message_count is positive, content MUST be retrieved from after and not including the specified timestamp or draft/msgid. If the message_count is negative, content MUST be retrieved from before the specified timestamp or draft/msgid.

The client is sending a known message as a starting point. If the oldest message a client has is at a particular time, this timestamp will be provided to CHATHISTORY and anything before it will be returned. The same goes for draft/msgid.

This works in the forward direction as well, in which the client knows the last message it has and wants to retrieve content from directly after it.

Theoretically, this works for any known message as a reference, and could be used to add context if a client wants to "jump" to a message, and then request X lines before and Y lines after.


## Limitations
Logging of messages and other actions must be enabled server-side and can be stored in any format desired, given appropriate software exists to retrieve and format such stored contnet. Scrollback can only be retrieved as far back as logs exist for the requesting user in the specified channel or query.

A method for securely identifying the requesting user must exist to ensure content is sent only to the appropriate users and clients. See below for more information.

## Security Considerations
Secure identification of users and clients must exist in order to ensure that users cannot obtain history that does not belong to them. Use of account names, internal account identifiers, or certificate fingerprints should be strongly considered when matching content to users. The server must verify the current user's identity matches that of the desired content. This information is not sent as part of the `CHATHISTORY` command and must be validated via other means, such as those stated above. Access must be checked first and return an `ACCESS_DENIED` error as appropriate. If no `ACCESS_DENIED` error exists, the server can check for returnable content.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Secure identification of users and clients must exist in order to ensure that users cannot obtain history that does not belong to them

I'm not sure this is that clear. Some history do not necessarily "belong" to an individual user. Could I not request the chat history for a public channel on a server?

If I'm requesting public information I do not think any identification of the user or client needs to exist, only that the client is registered with the server.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Secure identification of users and clients must exist in order to ensure that users cannot obtain history that was not previously available to them at the time the messages were originally sent.

Using the reworded version:

Could I not request the chat history for a public channel on a server?

While you can certainly request chathistory from a public channel, you should not be able to request history if you weren't in the channel at the time the messages were originally sent. Allowing users to obtain history for channels they never used or are no longer joined to would essentially allow people to "spy" on channels.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While you can certainly request chathistory from a public channel, you should not be able to request history if you weren't in the channel at the time the messages were originally sent.

What would the point of requesting history if I have already seen the history because I was in the channel?

I think this should be down to networks/implementors, as a network might decide that channels without +p (private) and +s (secret) have publicly available logs. Or some other reasons behind it.

Copy link
Author

@ghost ghost Apr 21, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you should not be able to request history if you weren't in the channel at the time the messages were originally sent.

What would the point of requesting history if I have already seen the history because I was in the channel?

This limitation exists to prevent retrieval of content you weren't in the channel for. If you were in there at the time the messages were originally sent, then that qualifies as content "previously available to [you]"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This limitation exists to prevent retrieval of content you weren't in the channel for.

Shouldn't this be a consideration on an IRC server and not rules enforced by the specification? An implementation of CHATHISTORY might want to provide public history for some channels. For example some private IRC server at a company where all employees could join later and view existing history.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. Will update to:

Secure identification of users and clients MUST exist in order to ensure that users cannot obtain history they are not authorized to view.


## Current Implementations
A ZNC-module implementation exists as [znc-chathistory](https://github.com/MuffinMedic/znc-chathistory). Client-side support for the ZNC module is in progress. Interest in supporting the batch type and command without ZNC has also been obtained, although specifics are not available at this time.

Note: This module may be out of date while the specification is reviewed and modified.

[batch]: http://ircv3.net/specs/extensions/batch-3.2.html
[batch/chathistory]: http://ircv3.net/specs/extensions/batch/chathistory-3.3.html
[server-time]: http://ircv3.net/specs/extensions/server-time-3.2.html
[draft/msgid]: https://github.com/ircv3/ircv3-specifications/pull/285
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[draft/labeled-response]: https://github.com/ircv3/ircv3-specifications/pull/162
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.