Return counts broken down by attribute from collection

[aaronmyhre]

Hey Percolate Team -

Is it possible to return the counts from publish-counts broken down by an attributes from the collection? I have an "events" collection with a user_id and Type and I would like to know the count of docs in the collection broken down by both. In sql terms select count(*) from events group by user_id, type.

Is this possible with publish-counts?

picture of desired-ish output

[tmeasday]

Hey @aaronmyhre, no, it's not really possible to do this right now (unless I guess you know all the ids before time that you want to publish.

I guess PCs would give you some building blocks to do that but it doesn't try at the moment.

Do you need this data to be reactive? Perhaps you are just looking for a mongo aggregation..

[aaronmyhre]

Hey @tmeasday -

Ok cool, I thought not, but just wanted to make sure.

The data doesn't really need to be reactive. While the user is on the page where these aggregate counts are available, I don't anticipate them changing. If thats the case, it sounds like I either denormalize the counts onto the user data set, or do an aggregation server side in mongo and publish the results as a new collection... I coming from a SQL world, so I am more hesitant about the former option, but it sounds like that is fairly normal in mongo...

[boxofrox]

Here's a possible implementation of groupBy, but there are some concerns.

Concerns

1. Security. This feature will likely create a means to leak data into the client environment that a user shouldn't have access to, because we currently lack a mechanism to filter out documents that shouldn't be counted for the current user.
2. Performance. groupBy may perform worse than now for basic counts due to the loss of cursor.count() (according to example below). Not sure where else we might find performance hits, yet.
3. Poor reactivity. If the count for one groupBy field changes, a simple implementation will have users updating all counts (visually or computationally) in the client, even though only one update/computation was needed.

Idea

• Add a groupBy option to specify how to select bins for counting.

  Meteor.publish('posts', function () {
    Counts.publish(this, 'postsByUser', Posts.find(), { groupBy: 'user_id' });
  });

• Convert count data-structure into an Object when groupBy is used.

  Counts.get('postsByUser');
  // => { 1: 5, 2: 3, 3: 6, ... }
  // object is a set of groupBy/count pairs, so the `groupBy` field type must
  // be usable as object keys.

• Update cursor observer to bin counts. We lose the cursor.count() optimization on basic counts.

  // general idea
  var count = groupBy ? {} : 0;
  var observer = {
    added: function (doc) {
      if (groupBy && doc.groupBy) {
        count[doc.groupBy] += countFn ? countFn(doc) : 1;
      }
  
      if (! initializing)
        self.changed('counts', name, { count: count });
    },
    changed: function (oldDoc, newDoc) {
      // use a more sophisticated guard here
      if (groupBy && oldDoc.groupBy && newDoc.groupBy) {
        count[doc.groupBy] += countFn ? (countFn(newDoc) - countFn(oldDoc)) : 0;
      }
  
      self.changed('counts', name, { count: count });
    },
    removed: function (doc) {
      if (groupBy && doc.groupBy) {
        count[doc.groupBy] -= countFn ? countFn(doc) : 1;
      }
  
      self.changed('counts', name, { count: count });
    }
  };

@aaronmyhre, you're welcome to fork the project and try it out. The code in publish counts is fairly simple. With this example, you'd have to run Counts.publish once for groupBy: 'user_id' and again for groupBy: 'type'.

[aaronmyhre]

Hey @boxofrox -

Very cool, thanks for the outline. I am traveling for the next four or five days, but when I settle down I will give this a whorl and let you know how it goes.

[Ajaxsoap]

I have the similar requirement as above, my use case is to count the documents by a user createdBy field, my use case is to count all the the user (branch users) documents based on the user company.

I've managed to achieved the counts of documents per company per branch based on user roles with the help of @boxofrox :) Thanks for that!

My requirement is something like this, if the role is HQ company hq(head office), it should display the document count of its branches.

What you see on the total enrollments column is actually the total number of documents the company has. I don't know how achieve this requirement, based on @tmeasday , it is not possible at this time.

Do you have any workaround for this?

[boxofrox]

@Ajaxsoap. I have an idea on this, but I'd rather not hijack @aaronmyhre's issue. Could you start a new issue with your post above, please?

[Ajaxsoap]

Sure, no problem. re-posted it here