Open Source Intelligence Virtual VM - OSINT4RIGHTS

VirtualBox VM based on Ubuntu 18.04.1 LTS (Bionic Beaver)

Install includes:

• Python3, pip3, tweepy, numpy, scipy, gensim, psycopg2, python3-opencv, face_recognition, websockets, pandas, matlibplot, sklearn, seaborn - https://www.python.org/
• R, R studio - https://www.rstudio.com/
• Ruby - https://www.ruby-lang.org/en/
• GRASS GIS - https://grass.osgeo.org/
• PostgreSQL, pgAdmin4, postgis, pgrouting - https://www.postgresql.org/
• NodeJS, NPM - https://nodejs.org/en/
• Node-Red (set up as background service) - https://github.com/node-red/node-red
  • access UI locally from web browser on 127.0.0.1:1880
  • help on core nodes found here: http://noderedguide.com/node-red-lecture-4-a-tour-of-the-core-nodes/
• Apache2 with CGI enabled - https://httpd.apache.org/
• Mapserver (CGI) - https://mapserver.org/
• PHP - http://www.php.net/
• Tensorflow for Python2.x - https://www.tensorflow.org/install/
• VLC (video player), GIMP (image manipulation), Audacity (sound edit tool)
• ffmpeg
• Dejavu audio tool - https://github.com/worldveil/dejavu

Computer Vision Applications Installed

• Places365 - https://github.com/CSAILVision/places365/blob/master/run_placesCNN_unified.py + http://places2.csail.mit.edu/demo.html
• YOLO - https://www.arunponnusamy.com/yolo-object-detection-opencv-python.html + https://timebutt.github.io/static/how-to-train-yolov2-to-detect-custom-objects/

Databases

• Geonames - download and install a local copy in PostgreSQL using Ruby script (/tools/other/ruby/download_geonames.rb) https://gist.github.com/kaspergrubbe/b0f87329319e6442d676c0edbf635be9

(or restore from pg backup file: geonames_backup)

VM folders:

/tools - for custom tools and scripts

/data - for language dictionaries, GIS data etc used by the tools

/workspace/input -- for files and images to be scanned

/workspace/output -- for output from running the tools

/workspace/tmp -- used by the tools in processing the files

Download Link & Instructions:

1) Download and Install Oracle VirtualBox (version 5.2.18+) for your operating system https://www.virtualbox.org/wiki/Downloads

2) Download and unzip the OSR4R VM file - [ download - 7GB zip file ] (ver 1.3) [changelog]

- put the hard drive image (.vdi) file somewhere on your local disk (e.g. in a new folder called c:/virtualmachines)

3) Open VirtualBox and add a new machine giving is the name OSINTLAB, configure it with 2+ CPUs and 8GB+ of RAM. Add a drive and point it to the .vdi file. Ensure you set it up for either mouse or tablet according to if you are using a laptop or desktop.

4) Double click on OSINTLAB in VirtualBox and wait for it to boot.

5) Log in as user osintlab with the password: $khf.t00ls$

Links and Technical Notes:

• https://github.com/node-red/node-red - set up to run as a background service (file: /etc/systemd/system/node-red.service) - connect from browser using 127.0.0.1:1880
• PostgreSQL database - username: postgres / password: osinttools
• pgAdmin4 setup as a background service (file: /etc/systemd/system/pgadmin4.service) - - access locally via web browser on 127.0.0.1:5050)
• mySQL database - username: root / password: osinttools