Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ported TcpReassembly example to C++11 #1229

Merged
merged 12 commits into from
Nov 9, 2023
Merged

Ported TcpReassembly example to C++11 #1229

Changes from 6 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
133d953
Ported TcpReassembly example to C++11
jpcofr Nov 4, 2023
d5dae20
Changed back a constant...
jpcofr Nov 4, 2023
93f9107
Change loop variable to constant reference
jpcofr Nov 4, 2023
4fa9f10
Use const reference for devList in listInterfaces function
jpcofr Nov 4, 2023
996a850
Attempt to fix C2059...
jpcofr Nov 4, 2023
86cf2c9
Attempt to fixed loop type in listInterfaces for windows target
jpcofr Nov 6, 2023
0e6b451
Change `interface` variable name to `dev`
seladb Nov 7, 2023
20ec477
Merge branch 'dev' into dev_port_C11_TcpReassemblyTests
seladb Nov 7, 2023
1852b18
Merge branch 'dev' into dev_port_C11_TcpReassemblyTests
jpcofr Nov 7, 2023
48be196
Rename `iter2` to `flow2`
seladb Nov 9, 2023
fbfaac9
More fixes
seladb Nov 9, 2023
a140bcf
Merge branch 'dev' into dev_port_C11_TcpReassemblyTests
seladb Nov 9, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 40 additions & 41 deletions Examples/TcpReassembly/main.cpp
View file
Open in desktop
jpcofr marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@

#include <stdlib.h>
#include <stdio.h>
#include <map>
#include <unordered_map>
#include <iostream>
#include <fstream>
#include <sstream>
Expand Down Expand Up @@ -53,8 +53,7 @@


// unless the user chooses otherwise - default number of concurrent used file descriptors is 500
#define DEFAULT_MAX_NUMBER_OF_CONCURRENT_OPEN_FILES 500

constexpr int DEFAULT_MAX_NUMBER_OF_CONCURRENT_OPEN_FILES = 500;

static struct option TcpAssemblyOptions[] =
{
Expand Down Expand Up @@ -83,7 +82,7 @@ class GlobalConfig
/**
* A private c'tor (as this is a singleton)
*/
GlobalConfig() { writeMetadata = false; writeToConsole = false; separateSides = false; maxOpenFiles = DEFAULT_MAX_NUMBER_OF_CONCURRENT_OPEN_FILES; m_RecentConnsWithActivity = nullptr; }
GlobalConfig() : m_RecentConnsWithActivity(nullptr), writeMetadata(false), writeToConsole(false), separateSides(false), maxOpenFiles(DEFAULT_MAX_NUMBER_OF_CONCURRENT_OPEN_FILES) { }

// A least-recently-used (LRU) list of all connections seen so far. Each connection is represented by its flow key. This LRU list is used to decide which connection was seen least
// recently in case we reached max number of open file descriptors and we need to decide which files to close
Expand Down Expand Up @@ -246,7 +245,7 @@ struct TcpReassemblyData
}

/**
* Clear all data (put 0, false or NULL - whatever relevant for each field)
* Clear all data (put 0, false or nullptr - whatever relevant for each field)
*/
void clear()
{
Expand Down Expand Up @@ -276,9 +275,8 @@ struct TcpReassemblyData
};


// typedef representing the connection manager and its iterator
typedef std::map<uint32_t, TcpReassemblyData> TcpReassemblyConnMgr;
typedef std::map<uint32_t, TcpReassemblyData>::iterator TcpReassemblyConnMgrIter;
// typedef representing the connection manager
typedef std::unordered_map<uint32_t, TcpReassemblyData> TcpReassemblyConnMgr;


/**
Expand Down Expand Up @@ -326,12 +324,13 @@ void printAppVersion()
*/
void listInterfaces()
{
const std::vector<pcpp::PcapLiveDevice*>& devList = pcpp::PcapLiveDeviceList::getInstance().getPcapLiveDevicesList();
auto const& devList = pcpp::PcapLiveDeviceList::getInstance().getPcapLiveDevicesList();

std::cout << std::endl << "Network interfaces:" << std::endl;
for (std::vector<pcpp::PcapLiveDevice*>::const_iterator iter = devList.begin(); iter != devList.end(); iter++)

for (auto& interface : devList)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@seladb The problem is still there. How do you check if the code compiles for different systems? Using Docker or directly on GitHub with its fast CI? I need a faster way to test changes; right now, I can test only 2 or 3 times in an hour.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I usually use VMs to test on different platforms. For unix/linux-based platforms you can also use docker.

For Windows you can use these free VMs offered by Microsoft:
https://developer.microsoft.com/en-us/windows/downloads/virtual-machines/

You can install VS 2019 / 2022 and test whatever you need

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jpcofr I think the issue is with the variable name... interface is probably a reserved word in Visual Studio 🤦‍♂️

I changed interface to dev and it seems to work now...

{
std::cout << " -> Name: '" << (*iter)->getName() << "' IP address: " << (*iter)->getIPv4Address().toString() << std::endl;
std::cout << " -> Name: '" << interface->getName() << "' IP address: " << interface->getIPv4Address().toString() << std::endl;
}
exit(0);
}
Expand All @@ -340,17 +339,17 @@ void listInterfaces()
/**
* The callback being called by the TCP reassembly module whenever new data arrives on a certain connection
*/
static void tcpReassemblyMsgReadyCallback(int8_t sideIndex, const pcpp::TcpStreamData& tcpData, void* userCookie)
static void tcpReassemblyMsgReadyCallback(const int8_t sideIndex, const pcpp::TcpStreamData& tcpData, void* userCookie)
{
// extract the connection manager from the user cookie
TcpReassemblyConnMgr* connMgr = (TcpReassemblyConnMgr*)userCookie;
auto connMgr = (TcpReassemblyConnMgr*)userCookie;

// check if this flow already appears in the connection manager. If not add it
TcpReassemblyConnMgrIter iter = connMgr->find(tcpData.getConnectionData().flowKey);
if (iter == connMgr->end())
auto flow = connMgr->find(tcpData.getConnectionData().flowKey);
if (flow == connMgr->end())
{
connMgr->insert(std::make_pair(tcpData.getConnectionData().flowKey, TcpReassemblyData()));
iter = connMgr->find(tcpData.getConnectionData().flowKey);
flow = connMgr->find(tcpData.getConnectionData().flowKey);
}

int8_t side;
Expand All @@ -362,9 +361,9 @@ static void tcpReassemblyMsgReadyCallback(int8_t sideIndex, const pcpp::TcpStrea
side = 0;

// if the file stream on the relevant side isn't open yet (meaning it's the first data on this connection)
if (iter->second.fileStreams[side] == nullptr)
if (flow->second.fileStreams[side] == nullptr)
{
// add the flow key of this connection to the list of open connections. If the return value isn't NULL it means that there are too many open files
// add the flow key of this connection to the list of open connections. If the return value isn't nullptr it means that there are too many open files
// and we need to close the connection with least recently used file(s) in order to open a new one.
// The connection with the least recently used file is the return value
uint32_t flowKeyToCloseFiles;
Expand All @@ -374,7 +373,7 @@ static void tcpReassemblyMsgReadyCallback(int8_t sideIndex, const pcpp::TcpStrea
if (result == 1)
{
// find the connection from the flow key
TcpReassemblyConnMgrIter iter2 = connMgr->find(flowKeyToCloseFiles);
auto iter2 = connMgr->find(flowKeyToCloseFiles);
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we rename the variable to flow2, the same way we renamed iter to flow?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 48be196

if (iter2 != connMgr->end())
{
// close files on both sides (if they're open)
Expand All @@ -397,25 +396,25 @@ static void tcpReassemblyMsgReadyCallback(int8_t sideIndex, const pcpp::TcpStrea
std::string fileName = GlobalConfig::getInstance().getFileName(tcpData.getConnectionData(), sideIndex, GlobalConfig::getInstance().separateSides) + ".txt";

// open the file in overwrite mode (if this is the first time the file is opened) or in append mode (if it was already opened before)
iter->second.fileStreams[side] = GlobalConfig::getInstance().openFileStream(fileName, iter->second.reopenFileStreams[side]);
flow->second.fileStreams[side] = GlobalConfig::getInstance().openFileStream(fileName, flow->second.reopenFileStreams[side]);
}

// if this messages comes on a different side than previous message seen on this connection
if (sideIndex != iter->second.curSide)
if (sideIndex != flow->second.curSide)
{
// count number of message in each side
iter->second.numOfMessagesFromSide[sideIndex]++;
flow->second.numOfMessagesFromSide[sideIndex]++;

// set side index as the current active side
iter->second.curSide = sideIndex;
flow->second.curSide = sideIndex;
}

// count number of packets and bytes in each side of the connection
iter->second.numOfDataPackets[sideIndex]++;
iter->second.bytesFromSide[sideIndex] += (int)tcpData.getDataLength();
flow->second.numOfDataPackets[sideIndex]++;
flow->second.bytesFromSide[sideIndex] += (int)tcpData.getDataLength();

// write the new data to the file
iter->second.fileStreams[side]->write((char*)tcpData.getData(), tcpData.getDataLength());
flow->second.fileStreams[side]->write((char*)tcpData.getData(), tcpData.getDataLength());
}


Expand All @@ -425,13 +424,13 @@ static void tcpReassemblyMsgReadyCallback(int8_t sideIndex, const pcpp::TcpStrea
static void tcpReassemblyConnectionStartCallback(const pcpp::ConnectionData& connectionData, void* userCookie)
{
// get a pointer to the connection manager
TcpReassemblyConnMgr* connMgr = (TcpReassemblyConnMgr*)userCookie;
auto connMgr = (TcpReassemblyConnMgr*)userCookie;

// look for the connection in the connection manager
TcpReassemblyConnMgrIter iter = connMgr->find(connectionData.flowKey);
auto connectionMngr = connMgr->find(connectionData.flowKey);

// assuming it's a new connection
if (iter == connMgr->end())
if (connectionMngr == connMgr->end())
{
// add it to the connection manager
connMgr->insert(std::make_pair(connectionData.flowKey, TcpReassemblyData()));
Expand All @@ -446,35 +445,35 @@ static void tcpReassemblyConnectionStartCallback(const pcpp::ConnectionData& con
static void tcpReassemblyConnectionEndCallback(const pcpp::ConnectionData& connectionData, pcpp::TcpReassembly::ConnectionEndReason reason, void* userCookie)
{
// get a pointer to the connection manager
TcpReassemblyConnMgr* connMgr = (TcpReassemblyConnMgr*)userCookie;
auto connMgr = (TcpReassemblyConnMgr*)userCookie;

// find the connection in the connection manager by the flow key
TcpReassemblyConnMgrIter iter = connMgr->find(connectionData.flowKey);
auto connection = connMgr->find(connectionData.flowKey);

// connection wasn't found - shouldn't get here
if (iter == connMgr->end())
if (connection == connMgr->end())
return;

// write a metadata file if required by the user
if (GlobalConfig::getInstance().writeMetadata)
{
std::string fileName = GlobalConfig::getInstance().getFileName(connectionData, 0, false) + "-metadata.txt";
std::ofstream metadataFile(fileName.c_str());
metadataFile << "Number of data packets in side 0: " << iter->second.numOfDataPackets[0] << std::endl;
metadataFile << "Number of data packets in side 1: " << iter->second.numOfDataPackets[1] << std::endl;
metadataFile << "Total number of data packets: " << (iter->second.numOfDataPackets[0] + iter->second.numOfDataPackets[1]) << std::endl;
metadataFile << "Number of data packets in side 0: " << connection->second.numOfDataPackets[0] << std::endl;
metadataFile << "Number of data packets in side 1: " << connection->second.numOfDataPackets[1] << std::endl;
metadataFile << "Total number of data packets: " << (connection->second.numOfDataPackets[0] + connection->second.numOfDataPackets[1]) << std::endl;
metadataFile << std::endl;
metadataFile << "Number of bytes in side 0: " << iter->second.bytesFromSide[0] << std::endl;
metadataFile << "Number of bytes in side 1: " << iter->second.bytesFromSide[1] << std::endl;
metadataFile << "Total number of bytes: " << (iter->second.bytesFromSide[0] + iter->second.bytesFromSide[1]) << std::endl;
metadataFile << "Number of bytes in side 0: " << connection->second.bytesFromSide[0] << std::endl;
metadataFile << "Number of bytes in side 1: " << connection->second.bytesFromSide[1] << std::endl;
metadataFile << "Total number of bytes: " << (connection->second.bytesFromSide[0] + connection->second.bytesFromSide[1]) << std::endl;
metadataFile << std::endl;
metadataFile << "Number of messages in side 0: " << iter->second.numOfMessagesFromSide[0] << std::endl;
metadataFile << "Number of messages in side 1: " << iter->second.numOfMessagesFromSide[1] << std::endl;
metadataFile << "Number of messages in side 0: " << connection->second.numOfMessagesFromSide[0] << std::endl;
metadataFile << "Number of messages in side 1: " << connection->second.numOfMessagesFromSide[1] << std::endl;
metadataFile.close();
}

// remove the connection from the connection manager
connMgr->erase(iter);
connMgr->erase(connection);
}


Expand Down
Loading