Request: External IP Filter List

[Matt-CyberGuy]

Happy New Years MC team, you guys are the best!

Got a quick, hopefully not difficult request. Our list of filtered IPs is growing more and more, and is difficult to interpret. Would it be possible to have the IP filter options reference a file with the list of IPs?

Added bonus would be if the IPs in the file could be listed in a way with a description of who the IP belongs to that would be ignored by MeshCentral.

[Ylianst]

Good idea, working on it now.

[Ylianst]

With the next version of MeshCentral, you can now do this in the config.json:

    "userAllowedIp": "file:userallowedips.txt",
    "userBlockedIp": "file:userblockedips.txt",
    "agentAllowedIp": "file:agentallowedips.txt"

You put file: followed by a file name that is in meshcentral-data. The file can look like this:

# My list of blocked IP's
185.101.70.0/24
185.46.85.0/24
37.9.44.0/24
37.9.45.0/24
5.189.205.0/24
5.189.206.0/24
5.189.207.0/24
5.62.153.0/24
5.62.156.0/24
5.62.158.0/24

# One more list
5.8.44.0/24
5.8.45.0/24
5.8.46.0/24
79.110.28.0/24
79.110.31.0/24
91.204.14.0/24
95.181.218.0/24
95.85.81.0/24

All the lines that start with a number or : will be used, everything else is ignored. So, you can put comments anyway you like, but probably best to start then with a # or something to make it clear.

This will be with MeshCentral v0.9.64, once published let me know if it works.

[asasin114]

Could I make one addition to this request? Could you make it so the allowed list accepts fqdn's? This would be super helpful for scenarios where a site has a WAN address issued by DHCP and is using Dynamic DNS services.

[Ylianst]

I am not sure why FQDN would be useful, mostly users and agents don't have FQDN's assigned to them and you can't specify a range of IP addresses using FQDN. These lists are for allowing or blocking IP addresses ranges that do not typically have FQDN's assigned to them. Unless I am missing something, not sure FQDN would be useful.

Also, if you block a FQDN instead of an IP address, someone can change the FQDN record and keep attacking?

[rapi3]

I want to filter login IP so I added my LAN network IP 192.168.1.0/24 to userallowedips.txt and I can't log in from LAN any more: Unauthorized
Latest version 0.9.91
I also use: ?key=xxx

Any ideas ?

[jsinoy]

Hi Ylianst ,

I am a big fan of meshcentral and it does really work well . I have tried the the solution above to filter ip address but unfortunately it doesn't work for me .

when my config.json looks like this , it works fine ,
"agentAllowedIP": "23.23.23.23",

however when my config.json looks like this , it doesn't work.
"agentAllowedIP": "file":agentallowedips.txt",

Note : agentallowedips.txt file is under meshcentral-data .

EDIT : it works when i have only one ip address entry in agentallowedips.txt file , but when i add another ip on the next line it wont work.

it works on:
agentallowedips.txt
23.24.24.52

but it wont work when the text file contains like this
24.24.24.52
24.24.24.53
23.45.66.7

[si458]

@silversword411 any chance you could add this to the docs in your free time?
also update the manifest as it still shows only an string or array
https://github.com/Ylianst/MeshCentral/blob/master/meshcentral-config-schema.json#L156-L159

[silversword411]

add this to the docs in your free time?

@si458 What exactly are you referring to? A writeup on how to block by IP?

[si458]

add this to the docs in your free time?

@si458 What exactly are you referring to? A writeup on how to block by IP?

Yes sorry basically this step in the docs somewhere

#3401 (comment)

[silversword411]

Actually, it's already there https://ylianst.github.io/MeshCentral/meshcentral/#server-ip-filtering

[si458]

Actually, it's already there https://ylianst.github.io/MeshCentral/meshcentral/#server-ip-filtering

thanks but i was talking about the fact you can use the file:banedips.txt value to load the ips from a text file

[silversword411]

Last posting said there was a problem with multiple IPs in the file, one per line.

I haven't tested, does it work for you?

[si458]

Last posting said there was a problem with multiple IPs in the file, one per line.

I haven't tested, does it work for you?

Works fine for me also same for @Ylianst
#4321 (comment)

[jsinoy]

Unfortunately, it really doesn't work to me . Not sure if i have misconfigured anything
Server version 1.0.63 running on linux on AWS (ec2) instance.

it works if my config.json looks like this;
"agentAllowedIp": "1.1.1.1,2.2.2.2,3.3.3.3"

however if i change it to this : it wont work anymore, agents are all offline and unreachable.
"agentAllowedIp": "file:agent_allowed_ips.txt"

agent_allowed_ips.txt file is located under meshcentral-data folder. As we a long list of ip address , it's much easier to track and add in a text file .

[silversword411]

Works fine for me also same

Can you give some screenshots of the text files, and config?

[CPINetworks]

I am not sure why FQDN would be useful, mostly users and agents don't have FQDN's assigned to them and you can't specify a range of IP addresses using FQDN. These lists are for allowing or blocking IP addresses ranges that do not typically have FQDN's assigned to them. Unless I am missing something, not sure FQDN would be useful.

Also, if you block a FQDN instead of an IP address, someone can change the FQDN record and keep attacking?

Allowing FQDN would allow us to white list tech's home IP's using DDNS. We do not have static IP's everywhere we wish to login from.

[jsinoy]

Works fine for me also same

Can you give some screenshots of the text files, and config?

[jsinoy]

When I add a single line in agentallowedips.txt file , it works , but when i add multiple lines of ip , it wont work anymore .

[silversword411]

I'm guessing the 122. is your test IP...what happens when you put it on the 2nd line?

Does it go from working to not working? Do you have to restart meshcentral for this to take effect, or is it live like the hosts file in windows?

[Matt-CyberGuy]

I know this is a slightly unrelated question, but once the multi-line IP filtering is solved for, can commented out lines be present in this file? We have a pretty long list of IPs, would love to be able to label them.

[Matt-CyberGuy]

oops, my bad... i just scrolled back thru the thread and saw it's supposed to be possible.

[Matt-CyberGuy]

I decided to try this out last night on a fresh install. It seems external lists are not working for me, even if I only have 1 ip. I have my list in the same directory as the config.json folder.

[silversword411]

https://ylianst.github.io/MeshCentral/meshcentral/#server-ip-filtering

[jsinoy]

I'm guessing the 122. is your test IP...what happens when you put it on the 2nd line?

Does it go from working to not working? Do you have to restart meshcentral for this to take effect, or is it live like the hosts file in windows?

Yes it works fine in a single line only like 1.1.1.1 in the txt file . However when i add multiple line , it wont work anymore. Restarted server few times as well.

It works when agentallowedips.txt content like this;
1.1.1.1

But it wont work when agentallowedips.txt content like this;
1.1.1.1
2.2.2..2

also tried this but still no luck
1.1.1.1/24
2.2.2.2/24

Maybe i have missed something ?

[jsinoy]

Hi Everyone,

I have tested in a new installed meshcentral-server in a windows machine , and ip filter in a text file is successful, I am not sure why it is not working in ec2-instance in AWS running linux . maybe i have missed something ?

[silversword411]

@jsinoy So it was a
Linux server: doesn't work
Windows server: works

for IP's in files problem?

[jsinoy]

@jsinoy So it was a Linux server: doesn't work Windows server: works

for IP's in files problem?

Yes , list of IP's in a text file . on a newly installed meshcentral-server on windows it is working , however on production server ec2-instance in AWS (linux) it doesn't work . im not sure maybe i have misconfigured something .

[Matt-CyberGuy]

Just revisited this, currently within linux this is still not functioning. I just spun up a brand new server. Once I change the ip filtering to a file, regardless of how many entries, it does not work. It does work if I just use a list within the config.json file.

For context, I'm also running this within a docker container behind nginx.

[si458]

@Matt-CyberGuy where are you storing the file?
can u share a quick snippet of your config.json?
and can u share a snippet of your file for example?

[Matt-CyberGuy]

@Matt-CyberGuy where are you storing the file? can u share a quick snippet of your config.json? and can u share a snippet of your file for example?

Below is my config.json. I've been slowly building it from scratch pulling from the config of an old instance I plan on retiring
The disabled allowedips.txt is located within the data directory with the config.json file.

{
  "$schema": "http://info.meshcentral.com/downloads/meshcentral-config-schema.json",
  "settings": {
    "cert": "remote.domain.com",
    "_WANonly": true,
    "_LANonly": true,
    "sessionKey": "",
    "port": 4430,
    "aliasPort": 443,
    "redirPort": 800,
    "_redirAliasPort": 80,
    "AgentPong": 300,
    "TLSOffload": "npmplus",
    "noagentupdate": true,
    "SelfUpdate": false,
    "AllowFraming": "false",
    "WebRTC": "true",
    "RelayDNS":[
        "1.rt.ITCompany.com",
        "2.rt.ITCompany.com",
        "3.rt.ITCompany.com"
        ],
    "AutoBackup": {
      "backupPath": "/opt/meshcentral/meshcentral-backups",
      "backupInvervalHours": 24,
      "keepLastDaysBackup": "30",
      "zippassword": ""
    }
  },
  "domains": {
        "": {
    "title": "ITCompany",
    "title2": "<br/>Cybersecurity | LAN | Data<h2>Remote Monitoring Portal</h2>",
    "welcomeText": "Please enter your email and given password above.<br/>If you are experiencing difficulty connecting to endpoints, need to add/remove users or systems, <br/>or you would like to have your password reset, please contact us at <a href='mailto:sos@ITCompany.com'>SoS@ITCompany.com</a>.",
    "titlePicture": "logo.png",
    "Footer": "<a target=_blank href='https://ITCompany.com'>ITCompany</a>",
    "mstsc": true,
    "agentCustomization": {
        "displayName": "ITCompany Remote Service",
        "description": "ITCompany Remote Service for remote monitoring, management and assistance.",
        "companyName": "ITCompany",
        "serviceName": "ITCompany Remote Service",
        "fileName": "ITCompanyRemote",
        "image": "agent-logo.png"
    },
    "agentFileInfo": {
        "icon":  "ITCompany-Remote.ico",
        "fileDescription": "ITCompany Remote Service for remote monitoring, management and assistance.",
        "legalCopyright":  "ITCompany Remote Agent (c)",
        "originalFilename":  "ITCompanyRemote.exe",
        "productName": "ITCompany Remote Service"
    },
    "altMessenging": {
        "name": "Video Chat",
        "url": "https://meet.ITCompany.com/{0}"
    },
    "terminal": {
        "launchCommand": {
        "linux": ""
        }
    },
    "authStrategies": {        
        "azure": {
            "newAccounts": true,
            "clientid": "",
            "clientsecret": "",
            "tenantid": ""
        },
        "google": {
            "newAccounts": true,
            "clientid": "",
            "clientsecret": ""
        }
    },
    "welcomePicture": "wallpaper.jpg",
    "welcomePictureFullScreen": true,
    "loginPicture": "welcome-logo.png",
    "minify": false,
    "NewAccounts": "true",
    "ManageAllDeviceGroups":["~azure:user@domain.com"],    
    "userNameIsEmail": true,
    "PasswordRequirements": { 
      "min": 8,
      "max": 64,
      "upper": 1,
      "lower": 1,
      "numeric": 1,
      "nonalpha": 1,
      "hint": true,
      "reset": 90,
      "force2factor": true
    },
    "_userAllowedIP": "file:allowedips.txt",
    "_userAllowedIP": "1.2.3.4",
    "certUrl": "https://remote.domain.com:443"
        }
  },
  "smtp": {
    "host": "smtp-relay.ITCompany.com",
    "port": 587,
    "from": "No-Reply | ITCompany Remote Portal <no-reply@ITCompany.com>",
    "user": "ITCompanygroup@gmail.com",
    "pass": "",
    "tls": false
},
  "_letsencrypt": {
    "__comment__": "Requires NodeJS 8.x or better, Go to https://letsdebug.net/ first before>",
    "_email": "myemail@mydomain.com",
    "_names": "myserver.mydomain.com",
        "production": true
  }
}

I know the agents and my remote connection are being relayed correctly through NGINX as the logs show the correct IP's... and as mentioned userAllowedIP functions correctly when using a list within config.json

[si458]

you have _userAllowedIP uncommented so its being ignored,
you need to remove the underscore _ and restart meshcentral

[Matt-CyberGuy]

you have _userAllowedIP uncommented so its being ignored, you need to remove the underscore _ and restart meshcentral

I'm aware, I had to disable it since I was not able to get in.

[si458]

ok strange one to try,
create the text file on a windows computer using notepad,
upload that file to your linux machine,
then restart meshcentral

the code shows
try { lines = obj.fs.readFileSync(obj.path.join(obj.datapath, arg.substring(5))).toString().split('\r\n').join('\r').split('\r'); } catch (ex) { }
so it loads the file from inside the meshcentral-config file
but also splits each line at the end with the windows return characters!
so its possibly a windows/linix line character issue!

[si458]

ok found an issue!!!
only the settings userAllowedIP loads from a text file,
the domains userAllowedIP doesnt read from a text file!
nice catch!
working on the bug/fix now!

[Matt-CyberGuy]

Oh that's awesome, I just finished trying every combination of making and loading the txt file from windows via my Mac... of course before seeing your response, so none of it worked :P

[si458]

ok commit done! df91c90
you can test in 15mins when the master docker image is built!

[Matt-CyberGuy]

This is awesome... I'm such a nerd :P
I just changed my docker image source to ghcr.io/ylianst/meshcentral:master
It's working for me now, and as far as I can tell, it doesn't matter what OS you make the list in.

[si458]

Glad it's fixed for you!
Fixed the not loading a file properly
Then fixed the separating each line if windows/linux!