Commit graph

121 commits

Author SHA1 Message Date
Carey Metcalfe ef151f8d28 Improve collector script and restructure code
- Moved the netflow library out of the src directory
- The UDP listener was restructured so that multiple threads can receive
  packets and push them into a queue. The main thread then pulls the
  packets off the queue one at a time and processes them. This means
  that the collector will never drop a packet because it was blocked on
  processing the previous one.
- Adds a property to the ExportPacket class to expose if any new
  templates are contained in it.
- The collector will now only retry parsing past packets when a new
  template is found. Also refactored the retry logic a bit to remove
  duplicate code (retrying just pushes the packets back into the main
  queue to be processed again like all the other packets).
- The collector no longer continually reads and writes to/from the disk.
  It just caches the data in memory until it exits instead.
2019-10-16 23:31:39 -04:00
Dominik Pataky ce2be709d6 Update README + LICENSE 2019-03-31 21:37:13 +02:00
Dominik Pataky 8de110980c Add tests for the collector (main.py). 2019-03-31 21:23:24 +02:00
Dominik Pataky 85e6af4bd2 Add buffering of exports with unknown template
Until now, exports which were received, but their template was not known,
resulted in KeyError exceptions due to a missing key in the template dict.
With this release, these exports are buffered until a template export
updates this dict, and all buffered exports are again examined.

Release v0.7.0

Fixes #4
Fixes #5
2019-03-31 20:51:34 +02:00
Dominik Pataky 5c7ec0aef8 Add additional field types (ASA, PANOS) and set fallback type to 0
refs #4 @ Github
2018-06-15 13:48:17 +02:00
Dominik Pataky 9395aafa71 Fix missing IP_PROTOCOL_VERSION field in analyzer
Checks for the key first and handles non-existence.
Update to Copyright notices.

Fixes #3
2018-02-20 12:09:54 +01:00
Dominik Pataky 691a3480fd Add duration to Connection 2017-10-29 19:38:33 +01:00
Dominik Pataky 6c267c8c77 Bump to 0.6; expand analyzer 2017-10-29 11:53:32 +01:00
Dominik Pataky 898d220a91 Add JSON export and analyzing example script 2017-10-28 19:00:18 +02:00
Dominik Pataky 92d8e724bf Fix merge for Python3 2017-10-28 17:34:55 +02:00
cookie 9df5bd426e
Merge pull request #2 from deeso/master
Created an installable Python Package
2017-10-28 17:19:29 +02:00
Adam Pridgen 23bc00a316 typo in logging message 2017-09-16 14:15:34 -05:00
Adam Pridgen e11105e950 added setup main file 2017-09-16 14:11:44 -05:00
Doм 7b24ae51e0 Merge pull request #1 from randerzander/master
Thanks for contributing @randerzander !
2016-12-12 18:46:06 +01:00
Randy Gelhausen bd22551669 converted hardcoded host/port to arg driven, switched int.from_bytes to Python2 friendly routine 2016-11-29 22:50:09 -05:00
Dominik Pataky 8fa999b877 Remove namedtuples import (old version) 2016-08-10 23:10:11 +02:00
Dominik Pataky aa2a8d8458 Add LICENSE and README.md 2016-08-10 22:47:35 +02:00
Dominik Pataky 546f96122f Fix datarecord saving bug; cleanup; license 2016-08-10 22:33:57 +02:00
Dominik Pataky 2d7c905d41 Parsing finished, bug in datarecord lists 2016-08-10 20:38:07 +02:00
Dominik Pataky 1be7552e06 Add classes 2016-08-10 18:55:38 +02:00
Dominik Pataky 6cf8356456 Basic implementation of udp socket listener and FlowRecord 2016-08-10 16:28:29 +02:00