Checklist/Collections for Ornaments

[joshhunt]

One thing that's quite difficult to show on DestinySets.com is what ornaments (and other resuablePlugItems) users have unlocked. My solution at the moment relies on the user having the 'host' item somewhere in their inventory, meaning many ornaments, especially on exotics, go missing on the site. Other reusablePlugItems, like Ghost Projections, suffer from this as well, but it's mitigated by the fact that users will always have a ghost in their inventory.

It would be very helpful if the API provided a way to tell me what ornaments (and other resuablePlugItems) users have unlocked, through a checklist or psuedo-collection, or similar.

[vthornheart-bng]

I dig that! I'll have to look into it, but on the surface that sounds like a reasonable thing that we could pull out!

[afpac]

Might be too late to chime in on this, but it would be rad to include Emblem Variants in this as well. I can pull this information right now through a very unpleasant series of queries against the Collectibles endpoint, but making the ornaments checklist an "item variants" checklist and including Emblems in it as well might make sense. If the checklist includes all reusablePlugItems like Josh suggested, it would hit Emblem Variants as well, but in case you don't take that full step, please consider including emblem variants along with ornaments.

[vthornheart-bng]

I dig that as well, if we can get the time to aggregate it together! Ideally, if I can find a way to automate getting you that data, that'll be the most likely way we can make it happen. I wouldn't be able to promise it if we end up having to upkeep manual configuration on our end... but my gut tells me that we'll be able to automate this, and if so then that makes it much more likely (when we get the time to do so!)

[joshhunt]

Whatever can be done would be greatly appreciated.

I'm hoping there's like a single source for unlockable reusablePlugItems somewhere that could be exposed as a checklist or something? 🤞

[vthornheart-bng]

Unfortunately not directly, but there may be some tricks I can do to infer it indirectly from the content. That's my hope at least!

[floatingatoll]

This would be another place where filtering unacquired ornaments for only those that could be acquired _may_ be a relevant concern.

…

On Jan 7, 2019, at 10:08, Vendal Thornheart ***@***.***> wrote: Unfortunately not directly, but there may be some tricks I can do to infer it indirectly from the content. That's my hope at least! — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[vthornheart-bng]

True, and that is both a good point and probably something that would be extremely difficult for me to automate. We have the ability to do that for Collectibles only because the designers explicitly added tests to each relevant collectible to determine visibility. These ornaments definitely won't have that information.

I'll need to give this some thought. The concerns about unacquired collectibles was a valid point, and this is too... and in this case I don't think I have an automate-able way to get that information (if I can even automate getting the list of them at all to begin with)

[ArkahnX]

Just chiming in, the game keeps track of what ornaments you own regardless of whether the item is in your inventory or in collections.

Shouldn't that mean there is some sort of database that holds ornament ownership status?

[vthornheart-bng]

Not directly - as in there's not a database of ornament ownership status as much as there is a set of either flags, values, or both that happen to hold the meaning of whether or not you own them. I think I've gone into it in other threads here, but the gist of Destiny's stateful data is that many - but not all - data is stored as generic values, either booleans or numeric values. Those values are used - either by content-driven systems or script-driven systems, and have no meaning aside from the systems that happen to either set or get those values: there's no single database that says "these values are used to track your ornaments" or "these values are used to track your current quest progress". It's a big bin of numbers and booleans, and which numbers and booleans correlate to which use cases depends entirely on what is using them.

When we expose that kind of data in the API, we're reverse engineering the meaning of these values by mining out where they're used and drawing conclusions based on the places where we see them being used, and what we know those places in content are used for. Whether I can get at the data (aka whether I can know which of those values are actually important in a programmatic way) and whether it is feasible to maintain whatever system I have to build that mines them out depends on many factors:

1. whether there's a single way that it's determined, or whether I'd need to aggregate how it's determined from multiple sources - and whether it will be likely that those sources/standards will be changed or added to in the future. The "home run" scenario is when there is just one standard, and that it is unlikely to change: that's how we get features like Collections, which were built with bulk consumption in mind and created by designers. The nightmare scenario is when there are many competing standards - because the game itself doesn't care how it's being done as long as the result is the same - and as a result they are constantly in flux, because unlike the API the game isn't looking to provide a consistent standard for how that data is accessed and used. That's how you end up with Advisors in Destiny 1, or Milestones in Destiny 2 - times where we've tried to standardize data that ultimately changed too frequently to be generalized at the level at which I attempted to generalize it. (A corollary is when existing "bins" of data are deprecated for their original use, but then are re-used in new ways for new purposes without our knowledge that such a change took place. That's how things like our mis-reporting of Season Ranks at the start of season 5 came about, as an interesting concrete example)

2. Whether any of the values used are determined by sources that aren't content driven (and thus I'd need to maintain the relationship manually: I feel that this is unlikely in this scenario, but I can't rule it out without investigation)

3. Whether I'll be able to determine not just whether you've collected the item, but also whether or not the item is still collectible (this goes back to the problem @floatingatoll mentioned, and I feel that it's very unlikely that this data is going to be available, which could potentially scuttle the idea)

When you see something like Collections, what you're seeing is a place where designers made an explicit decision to make a standardized spot in content - the "Collectible definitions" that ultimately become DestinyCollectibleDefinitions, along with the DestinyPresentationNodeDefinitions - where every meaningful flag and value is used in a highly standardized and predictable way, and that way was set as the canonical standard for how those flags and values are used from there on out. It makes our job of mining out that data much easier, but unfortunately the only time when we get data that's that clean is when they want to/have time to provide a similar "browsing" feature in the game. When there's not a direct need - or insufficient time - to do that kind of work (and the extra UI etc.. work it implies for other systems in the game to use it), you end up with systems where someone looking to make a "browsing" feature for content has to pull together multiple, often changing sources (and hope that none of those sources are driven by something un-scrapable, like scripts).

It should be noted as well that often attempts to standardize some of the uglier "anything goes" "systems" resulted in streamlined and standardized systems that were, for lack of a better description, "more boring". Like the initial standardization of Milestones at D2 launch, or the vendors-and-tokens system of rewards. Those are examples of systems that provided one single way to do things; that was great for creating and mining content, but bad for providing a variety of user experiences. It's worth having it be more of a hodge-podge for the sake of providing variety, so know that the lack of standardization isn't necessarily a bad thing! Just something that makes the work of the API a bit harder. :)

It's a jungle in there!

[vthornheart-bng]

(a good, specific example of how these flags and values are often unpredictable is actually something you can see in collections itself: if you start a new account, pay attention to when all of the green and blue items unlock. You'll see that most of them unlock at once, in response to a single story mission being completed or level being reached, regardless of whether or not you actually acquired them! It's a clever re-use of existing state by the folks who designed the Collections, to try and make a best guess as to when people would have been likely to see all these items: and in doing so, they saved a ton of storage for items that people more-or-less don't care about: but it also points out just how freely designers can associate state with behaviors in Destiny's content, even in a more "strictly defined" system like Collections! You can imagine from there how more open-ended systems might behave.)

[ArkahnX]

I definitely remember this discussion before!

I can't say I wasn't hoping for a single easy to read spot for all of that data, but it definitely doesnt seem to be the case.

But, it did bring about another awesome "vthorn story" so I'd say that's a win

[vthornheart-bng]

Aye - I wish it was as well! Though I am grateful that as much of this kind of evaluation exists in content as it does (as opposed to scripts). I sometimes think about that - how much harder it would have been to have an API if all of this stuff happened/was defined in scripts instead of it being defined in content. How much more manual curation we'd have had to have, how even more difficult it would have been to keep up with how systems worked and interacted with each other in order to expose it in a standardized way. It's interesting how that initial engineering/design decision shaped the way the game was created - and in many ways, made the API something that could feasibly be worked on by just a small number of people (often one... though I really wish sometimes that there was more than one, or at least one doing it full time ;) I digress!).

[ckhicks]

We're glad you're the one...even if we all wish there were more, for your sake. XD