Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TIP] Add IntelX mixin #58

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

[TIP] Add IntelX mixin #58

wants to merge 4 commits into from

Conversation

nullJaX
Copy link

@nullJaX nullJaX commented Dec 21, 2019

This PR aims to provide basic integration with IntelX service. It uses specific mixin to communicate with a service. API key and domain URL will be both stored in API key storage. Currently in a test phase.

The document used during the implementation: https://github.com/IntelligenceX/SDK/blob/master/Intelligence%20X%20API.pdf
I have also used example code from here: https://github.com/IntelligenceX/SDK/tree/master/Python

@nullJaX
Copy link
Author

nullJaX commented Dec 21, 2019

Also, this is a part of: lanmaster53/recon-ng-marketplace#80

@nullJaX nullJaX changed the title [WIP] Add IntelX mixin [TIP] Add IntelX mixin Jan 4, 2020
@lanmaster53
Copy link
Owner

lanmaster53 commented Jan 13, 2020

This is really good work, but I hesitate adding it to the core framework due to the increased technical debt. What happens when they decide to change their stuff? The entire reason I created the marketplace was to prevent this exact thing. So, while this code is good and deserves to be seen and used, let's explore how we could make this happen through the marketplace as opposed to the core. Any ideas?

I realize that there are already some mixins like this, but I would like to move those as well and only provide core mixins with the core. Perhaps we add a mixin list to the marketplace meta data, and when a module gets installed, the associated module mixin gets installed with it. Thoughts? Better idea?

@nullJaX
Copy link
Author

nullJaX commented Jan 14, 2020

This is indeed a good idea to split mixins into two categories: framework specific (ie. threading) that will be stored in this repo and external mixins (ie. github) that will be downloaded from marketplace, stored in mixins directory in root. The content of this directory would be downloaded always (no matter if user downloads specific modules or whole repository). What do you think, @lanmaster53 ?

@lanmaster53
Copy link
Owner

This is indeed a good idea to split mixins into two categories: framework specific (ie. threading) that will be stored in this repo and external mixins (ie. github) that will be downloaded from marketplace, stored in mixins directory in root. The content of this directory would be downloaded always (no matter if user downloads specific modules or whole repository). What do you think, @lanmaster53 ?

I think the mixins need to be downloaded when needed, not by default. Pretty much treated as trusted dependencies. It doesn't need to be done explicitly by the user though. The module itself can have metadata that tells the framework that it requires a mixin, and then the framework can install it along side the module. Doing them together ensures that the mixin is there when needed, otherwise managing a decoupled dependency could cause headaches. Also, the mixins need to be written in use space. They cannot be written to the application folder. So there's that too.

@lanmaster53
Copy link
Owner

This is indeed a good idea to split mixins into two categories: framework specific (ie. threading) that will be stored in this repo and external mixins (ie. github) that will be downloaded from marketplace, stored in mixins directory in root. The content of this directory would be downloaded always (no matter if user downloads specific modules or whole repository). What do you think, @lanmaster53 ?

I think the mixins need to be downloaded when needed, not by default. Pretty much treated as trusted dependencies. It doesn't need to be done explicitly by the user though. The module itself can have metadata that tells the framework that it requires a mixin, and then the framework can install it along side the module. Doing them together ensures that the mixin is there when needed, otherwise managing a decoupled dependency could cause headaches. Also, the mixins need to be written in use space. They cannot be written to the application folder. So there's that too.

I just realized how difficult this is going to be if the mixin has its own dependencies... I gotta table this. I don't think I want to go down this path. Perhaps you can make a pip module out of your code and add it as a dependency to the modules you want to use with it. Would that make sense, or is there too much code that is customized to work with just Recon-ng in the mixin?

@nullJaX
Copy link
Author

nullJaX commented Jan 14, 2020

Well, from the programming point of view, mixins are simply base classes that are inherited. I suppose that it can be placed in marketplace anyway as a base class for all intelx related stuff. The downside is that it would break the current marketplace directory structure (intelx/domains-domains.py instead of domains-domains/intelx.py).

Speaking of a structure, after IntelX SDK investigation, endpoints return various data for an entry. Given a domain, it may return both email addresses and other domains (or even contacts). In this case I was planning to make a directory intelx that would store python files with only one selector in their names. These names would determine what the input selector type is. For example:
domains.py would send domains but write to database various results that recon-ng supports. Is it a good idea, @lanmaster53 ?

@Kleissner
Copy link

We have released a completely rewritten Python API wrapper in our SDK: https://github.com/IntelligenceX/SDK/tree/master/Python

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants