Skip to content

Commit 4f36c92

Browse files
StefanSaP1llus
StefanSa
and
P1llus
authored
sophosxg-module initial release (elastic#19591)
What does this PR do? This PR Introduces the SophosXG filebeat module. Focusing currently on SophosXG Firewall, but should include other Sophos products as separate PR's later on. Why is it important? Adding more supported products to the filebeat portfolio. Co-authored-by: StefanSa <StefanSa@users.noreply.github.com> Co-authored-by: P1llus <pillus@chasenet.org>
1 parent 1487527 commit 4f36c92

46 files changed

Lines changed: 12099 additions & 0 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎filebeat/docs/fields.asciidoc‎

Lines changed: 1881 additions & 0 deletions
Large diffs are not rendered by default.

‎filebeat/docs/modules/sophosxg.asciidoc‎

Lines changed: 159 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,159 @@
1+
////
2+
This file is generated! See scripts/docs_collector.py
3+
////
4+
5+
[[filebeat-module-sophosxg]]
6+
[role="xpack"]
7+
8+
:modulename: sophosxg
9+
:has-dashboards: false
10+
11+
== SophosXG module
12+
13+
This is a module for SophosXG SFOS logs sent in the syslog format.
14+
15+
To configure a remote syslog destination, please reference the https://community.sophos.com/kb/en-us/123184[SophosXG/SFOS Documentation].
16+
17+
The syslog format choosen should be `Default`.
18+
19+
include::../include/gs-link.asciidoc[]
20+
21+
[float]
22+
=== Compatibility
23+
24+
This module has been tested against SFOS version 17.5.x and 18.0.x.
25+
Versions above this are expected to work but have not been tested.
26+
27+
include::../include/configuring-intro.asciidoc[]
28+
29+
:fileset_ex: firewall
30+
31+
include::../include/config-option-intro.asciidoc[]
32+
33+
[float]
34+
==== `firewall` fileset settings
35+
36+
[source,yaml]
37+
----
38+
- module: sophosxg
39+
firewall:
40+
enabled: true
41+
var.input: udp
42+
var.syslog_host: 0.0.0.0
43+
var.syslog_port: 9005
44+
var.host_name: firewall.localgroup.local
45+
----
46+
47+
include::../include/var-paths.asciidoc[]
48+
49+
*`var.input`*::
50+
51+
The input to use, can be either the value `tcp`, `udp` or `file`.
52+
53+
*`var.syslog_host`*::
54+
55+
The interface to listen to all syslog traffic. Defaults to localhost.
56+
Set to 0.0.0.0 to bind to all available interfaces.
57+
58+
*`var.syslog_port`*::
59+
60+
The port to listen for syslog traffic. Defaults to 9005.
61+
62+
*`var.host_name`*::
63+
64+
Host name / Observer name, since SophosXG does not provide this in the syslog file.
65+
Default to `firewall.localgroup.local`
66+
67+
68+
[float]
69+
==== SophosXG ECS fields
70+
71+
This is a list of FortiOS fields that are mapped to ECS.
72+
73+
[options="header"]
74+
|==============================================================
75+
| SophosXG Fields | ECS Fields |
76+
| application | network.protocol |
77+
| classification | rule.category |
78+
| device_id | observer.serial_number |
79+
| domainname | url.domain |
80+
| dst_host | destination.address |
81+
| dst_int | observer.egress.interface.name |
82+
| dstzonetype | observer.egress.zone |
83+
| dst_ip | destination.ip |
84+
| destinationip | destination.ip |
85+
| dst_mac | destination.mac |
86+
| dstname | destination.address |
87+
| dst_port | destination.port |
88+
| dst_domainname | url.domain |
89+
| duration | event.duration |
90+
| filename | file.name |
91+
| filetype | file.extension |
92+
| file_size | file.size |
93+
| file_path | file.directory |
94+
| fw_rule_id | rule.id |
95+
| from_email_address | source.user.email |
96+
| httpstatus | http.response.status_code |
97+
| in_interface | observer.ingress.interface.name|
98+
| log_id | event.code |
99+
| log_subtype | event.action |
100+
| message | message |
101+
| method | http.request.method |
102+
| policy_type | rule.ruleset |
103+
| protocol | network.transport |
104+
| recv_bytes | destination.bytes |
105+
| recv_pkts | destination.packets |
106+
| referer | http.request.referrer |
107+
| sent_bytes | source.bytes |
108+
| sent_pkts | source.packets |
109+
| sha1sum | file.hash.sha1 |
110+
| srczonetype | observer.ingress.zone |
111+
| src_ip | source.ip |
112+
| src_domainname | url.domain |
113+
| sourceip | source.ip |
114+
| src_mac | source.mac |
115+
| src_port | source.port |
116+
| status_code | http.response.status_code |
117+
| time_zone | event.timezone |
118+
| to_email_address | destination.user.email |
119+
| tran_dst_ip | destination.nat.ip |
120+
| tran_dst_port | destination.nat.port |
121+
| tran_src_ip | source.nat.ip |
122+
| tran_src_port | source.nat.port |
123+
| url | url.original |
124+
| user_agent | user_agent.original |
125+
| useragent | user_agent.original |
126+
| user_gp | source.user.group |
127+
| user_name | source.user.name |
128+
| ws_protocol | http.version |
129+
|==============================================================
130+
131+
:has-dashboards!:
132+
133+
:fileset_ex!:
134+
135+
[float]
136+
[[dynamic-script-compilations]]
137+
=== Dynamic Script Compilations
138+
139+
The `sophosxg` filesets are based on Elasticsearch ingest pipelines and
140+
make extensive use of script processors and painless conditions. This can cause
141+
the pipelines to fail loading the first time the module is used, due to exceeding
142+
the maximum script compilation limits. It is recommended to tune the following
143+
parameters on your Elasticsearch cluster:
144+
145+
- {ref}/circuit-breaker.html#script-compilation-circuit-breaker[script.max_compilations_rate]:
146+
Increase to at least `100/5m`.
147+
148+
- {ref}/modules-scripting-using.html#modules-scripting-using-caching[script.cache.max_size]:
149+
Increase to at least `300` if using both filesets or other script-heavy modules.
150+
151+
:modulename!:
152+
153+
154+
[float]
155+
=== Fields
156+
157+
For a description of each field in the module, see the
158+
<<exported-fields-sophosxg,exported fields>> section.
159+

‎filebeat/docs/modules_list.asciidoc‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,7 @@ This file is generated! See scripts/docs_collector.py
5555
* <<filebeat-module-redis>>
5656
* <<filebeat-module-santa>>
5757
* <<filebeat-module-sonicwall>>
58+
* <<filebeat-module-sophosxg>>
5859
* <<filebeat-module-squid>>
5960
* <<filebeat-module-suricata>>
6061
* <<filebeat-module-system>>
@@ -120,6 +121,7 @@ include::modules/rapid7.asciidoc[]
120121
include::modules/redis.asciidoc[]
121122
include::modules/santa.asciidoc[]
122123
include::modules/sonicwall.asciidoc[]
124+
include::modules/sophosxg.asciidoc[]
123125
include::modules/squid.asciidoc[]
124126
include::modules/suricata.asciidoc[]
125127
include::modules/system.asciidoc[]

‎x-pack/filebeat/filebeat.reference.yml‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1335,6 +1335,30 @@ filebeat.modules:
13351335
# "+02:00" for GMT+02:00
13361336
# var.tz_offset: local
13371337

1338+
#------------------------------- Sophosxg Module -------------------------------
1339+
- module: sophosxg
1340+
firewall:
1341+
enabled: true
1342+
1343+
# Set which input to use between tcp, udp (default) or file.
1344+
#var.input: udp
1345+
1346+
# The interface to listen to syslog traffic. Defaults to
1347+
# localhost. Set to 0.0.0.0 to bind to all available interfaces.
1348+
#var.syslog_host: localhost
1349+
1350+
# The port to listen for syslog traffic. Defaults to 9004.
1351+
#var.syslog_port: 9005
1352+
1353+
# firewall default hostanme
1354+
#var.default_host_name: firewall.localgroup.local
1355+
1356+
# known firewalls
1357+
#var.known_devices:
1358+
# "device1_serialnumber": "a.host.local"
1359+
# "device2_serialnumber": "b.host.local"
1360+
1361+
13381362
#-------------------------------- Squid Module --------------------------------
13391363
- module: squid
13401364
log:

‎x-pack/filebeat/include/list.go‎

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎x-pack/filebeat/module/sophosxg/_meta/config.yml‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
- module: sophosxg
2+
firewall:
3+
enabled: true
4+
5+
# Set which input to use between tcp, udp (default) or file.
6+
#var.input: udp
7+
8+
# The interface to listen to syslog traffic. Defaults to
9+
# localhost. Set to 0.0.0.0 to bind to all available interfaces.
10+
#var.syslog_host: localhost
11+
12+
# The port to listen for syslog traffic. Defaults to 9004.
13+
#var.syslog_port: 9005
14+
15+
# firewall default hostanme
16+
#var.default_host_name: firewall.localgroup.local
17+
18+
# known firewalls
19+
#var.known_devices:
20+
# "device1_serialnumber": "a.host.local"
21+
# "device2_serialnumber": "b.host.local"
22+

‎x-pack/filebeat/module/sophosxg/_meta/docs.asciidoc‎

Lines changed: 146 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,146 @@
1+
[role="xpack"]
2+
3+
:modulename: sophosxg
4+
:has-dashboards: false
5+
6+
== SophosXG module
7+
8+
This is a module for SophosXG SFOS logs sent in the syslog format.
9+
10+
To configure a remote syslog destination, please reference the https://community.sophos.com/kb/en-us/123184[SophosXG/SFOS Documentation].
11+
12+
The syslog format choosen should be `Default`.
13+
14+
include::../include/gs-link.asciidoc[]
15+
16+
[float]
17+
=== Compatibility
18+
19+
This module has been tested against SFOS version 17.5.x and 18.0.x.
20+
Versions above this are expected to work but have not been tested.
21+
22+
include::../include/configuring-intro.asciidoc[]
23+
24+
:fileset_ex: firewall
25+
26+
include::../include/config-option-intro.asciidoc[]
27+
28+
[float]
29+
==== `firewall` fileset settings
30+
31+
[source,yaml]
32+
----
33+
- module: sophosxg
34+
firewall:
35+
enabled: true
36+
var.input: udp
37+
var.syslog_host: 0.0.0.0
38+
var.syslog_port: 9005
39+
var.host_name: firewall.localgroup.local
40+
----
41+
42+
include::../include/var-paths.asciidoc[]
43+
44+
*`var.input`*::
45+
46+
The input to use, can be either the value `tcp`, `udp` or `file`.
47+
48+
*`var.syslog_host`*::
49+
50+
The interface to listen to all syslog traffic. Defaults to localhost.
51+
Set to 0.0.0.0 to bind to all available interfaces.
52+
53+
*`var.syslog_port`*::
54+
55+
The port to listen for syslog traffic. Defaults to 9005.
56+
57+
*`var.host_name`*::
58+
59+
Host name / Observer name, since SophosXG does not provide this in the syslog file.
60+
Default to `firewall.localgroup.local`
61+
62+
63+
[float]
64+
==== SophosXG ECS fields
65+
66+
This is a list of FortiOS fields that are mapped to ECS.
67+
68+
[options="header"]
69+
|==============================================================
70+
| SophosXG Fields | ECS Fields |
71+
| application | network.protocol |
72+
| classification | rule.category |
73+
| device_id | observer.serial_number |
74+
| domainname | url.domain |
75+
| dst_host | destination.address |
76+
| dst_int | observer.egress.interface.name |
77+
| dstzonetype | observer.egress.zone |
78+
| dst_ip | destination.ip |
79+
| destinationip | destination.ip |
80+
| dst_mac | destination.mac |
81+
| dstname | destination.address |
82+
| dst_port | destination.port |
83+
| dst_domainname | url.domain |
84+
| duration | event.duration |
85+
| filename | file.name |
86+
| filetype | file.extension |
87+
| file_size | file.size |
88+
| file_path | file.directory |
89+
| fw_rule_id | rule.id |
90+
| from_email_address | source.user.email |
91+
| httpstatus | http.response.status_code |
92+
| in_interface | observer.ingress.interface.name|
93+
| log_id | event.code |
94+
| log_subtype | event.action |
95+
| message | message |
96+
| method | http.request.method |
97+
| policy_type | rule.ruleset |
98+
| protocol | network.transport |
99+
| recv_bytes | destination.bytes |
100+
| recv_pkts | destination.packets |
101+
| referer | http.request.referrer |
102+
| sent_bytes | source.bytes |
103+
| sent_pkts | source.packets |
104+
| sha1sum | file.hash.sha1 |
105+
| srczonetype | observer.ingress.zone |
106+
| src_ip | source.ip |
107+
| src_domainname | url.domain |
108+
| sourceip | source.ip |
109+
| src_mac | source.mac |
110+
| src_port | source.port |
111+
| status_code | http.response.status_code |
112+
| time_zone | event.timezone |
113+
| to_email_address | destination.user.email |
114+
| tran_dst_ip | destination.nat.ip |
115+
| tran_dst_port | destination.nat.port |
116+
| tran_src_ip | source.nat.ip |
117+
| tran_src_port | source.nat.port |
118+
| url | url.original |
119+
| user_agent | user_agent.original |
120+
| useragent | user_agent.original |
121+
| user_gp | source.user.group |
122+
| user_name | source.user.name |
123+
| ws_protocol | http.version |
124+
|==============================================================
125+
126+
:has-dashboards!:
127+
128+
:fileset_ex!:
129+
130+
[float]
131+
[[dynamic-script-compilations]]
132+
=== Dynamic Script Compilations
133+
134+
The `sophosxg` filesets are based on Elasticsearch ingest pipelines and
135+
make extensive use of script processors and painless conditions. This can cause
136+
the pipelines to fail loading the first time the module is used, due to exceeding
137+
the maximum script compilation limits. It is recommended to tune the following
138+
parameters on your Elasticsearch cluster:
139+
140+
- {ref}/circuit-breaker.html#script-compilation-circuit-breaker[script.max_compilations_rate]:
141+
Increase to at least `100/5m`.
142+
143+
- {ref}/modules-scripting-using.html#modules-scripting-using-caching[script.cache.max_size]:
144+
Increase to at least `300` if using both filesets or other script-heavy modules.
145+
146+
:modulename!:

‎x-pack/filebeat/module/sophosxg/_meta/fields.yml‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
- key: sophosxg
2+
title: "sophosxg"
3+
description: >
4+
sophosxg Module
5+
fields:
6+
- name: sophosxg
7+
type: group
8+
description: >
9+
fields:

0 commit comments

Comments
 (0)