티스토리 뷰

Cloud/Cloud Native

Falco

jacobbaek Jacob_baek 2021. 1. 3. 15:15

Falco

오픈소스 실시간 보안 도구로 정의된 특정 행위에 대해 모니터링하는도구이다.

2018년도 부터 시작되었고 sysdig에 의해 CNCF에 기여되었다. 현재는 incubating 과정에 있는 project이다.

간단하게 동작원리를 설명하자면,
커널 이벤트 모니터링하고 사전에 정의해놓은 rule에 행위상 위반되는 상황이 발생되면 경고를 발생시킨다.

Falco는 Kubernetes, Linux, Cloud-Native 대상의 보안 rule set을 정의하여 사용할수 있다.

Falco가 하는 일

Falco는 시스템/서비스 등 을 모니터링하고 안전하게 만드는 다음과 같은 system call을 사용한다.

  • Kernel로부터 실시간 Linux System call들을 Parsing
  • 강력한 정책(rule) 엔진을 사용해 stream을 확고히 함
  • 정책(rule) 위반시 경고(alert) 발생

용어

여기서 이야기되는 용어에 대해 간단히 집고 넘어가자.

rules(정책)
Falco의 설정파일로 정의되며 시스템상에 확인할 이벤트를 대변한다.
즉, Falco에 의해 모니터링할 항목들을 정의한 설정파일이다.
YAML 기반의 파일로 rule을 정의할수 있다.

alerts(경고)
경고를 아래와 같은 방식으로 출력할수 있다.

  • 표준 출력(standard output)
  • file
  • syslog
  • program 실행
  • http[s] end point 호출
  • gRPC API를 통한 client 호출

구성

다음 3가지 주요 구성요소가 존재한다.

  • userspace program : cli 도구인 falco
  • configuration : Falco가 어떻게 동작할지 정의하는 rule
  • driver :
    아래 항목들중 하나가 사용된다.
    • C++ library인 libscap와 libsinsp로 만들어진 kernel module (default : 즉 별도의 option없이 실행되는경우 기본 kernel module 사용)
    • BPF probe (default가 아닌 대안으로 사용되어진다. 또한 최신 커널에서만 eBPF probe를 사용할수 있다.)
    • userspace에서 probe를 수행(userspace 에서 100% 동작되는 모드)
    • 출처 : https://falco.org/docs/event-sources/drivers/

다음 installation 항목에서 소개되겠지만 kernel-header를 설치하고 사용할 수 있다.
이말인즉슨, custom kernel module을 생성한다는 의미이다. (실제 falco.ko가 load된다.)

detection

가장 중요한 부분일거라 판단된다.
falco를 통해 취약점을 찾긴할텐데 어떤항목까지 찾아낼지에 대한 부분이다.
Falco는 Linux system call들에 포함된 어떤 행위에 대해 위협을 찾아내고 알람을 발생시킬수 있다.

  • kubernetes상에 container나 pod에서 동작되는 shell

  • privileged mode에서 동작되거나 민감한 경로(ex. /proc)를 mount한 container

  • 예상하지 못한 type의 child process를 가진 parent 서버 process

  • 예상하지 못한 민감 파일에 대한 read (ex. /etc/shadow)

  • device file이 아님에도 /dev와 같은 device file에 write를 시도

  • 표준 system binary 중에 outbound network connection을 가지지 않아야 하는 binary가 outbound connection을 시도

  • privileged pod가 kubernetes cluster안에서 시작

  • 출처 : https://github.com/falcosecurity/falco#what-can-falco-detect

좀더 자세한 예제는 다음 링크를 참고

Falco 사용법

installation

아래는 ubuntu 20.04 기준으로 작성한 내용이며 rpm,deb,binary 형태로 제공하고 있기에 입맛에 맞게 설치하여 사용하면 된다.

root@jacob-laptop:/home/jacob# curl -s https://falco.org/repo/falcosecurity-3672BA8F.asc | apt-key add -
root@jacob-laptop:/home/jacob# echo "deb https://dl.bintray.com/falcosecurity/deb stable main" | tee -a /etc/apt/sources.list.d/falcosecurity.list
root@jacob-laptop:/home/jacob# apt-get update -y
root@jacob-laptop:/home/jacob# apt install -y falco
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages were automatically installed and are no longer required:
  golang-1.13-go golang-1.13-race-detector-runtime golang-1.13-src golang-race-detector-runtime golang-src
Use 'apt autoremove' to remove them.
The following NEW packages will be installed:
  falco
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 5,304 kB of archives.
After this operation, 14.8 MB of additional disk space will be used.
Get:1 https://dl.bintray.com/falcosecurity/deb stable/main amd64 falco amd64 0.26.2 [5,304 kB]
Fetched 5,304 kB in 2s (2,856 kB/s)
Selecting previously unselected package falco.
(Reading database ... 305460 files and directories currently installed.)
Preparing to unpack .../falco_0.26.2_amd64.deb ...
Unpacking falco (0.26.2) ...
Setting up falco (0.26.2) ...
Loading new falco-2aa88dcf6243982697811df4c1b484bcbe9488a2 DKMS files...
Building for 5.4.0-58-generic
Building initial module for 5.4.0-58-generic
Secure Boot not enabled on this system.
Done.

falco.ko:
Running module version sanity check.
 - Original module
   - No original module exists within this kernel
 - Installation
   - Installing to /lib/modules/5.4.0-58-generic/updates/dkms/

depmod....

DKMS: install completed.
Processing triggers for systemd (245.4-4ubuntu3.3) ...
root@jacob-laptop:/home/jacob# lsmod | grep falco
root@jacob-laptop:/home/jacob# modprobe falco
root@jacob-laptop:/home/jacob# lsmod | grep falco
falco                 638976  0

참고로 위와 같이 custom kernel module인 falco가 load됨을 확인할수 있다.

Usage

사용법은 falco cli에 대한 부분이라 먼저 help message를 확인해보자.

jacob@jacob-laptop:~/workspaces$ falco --help
Falco version: 0.26.2
Usage: falco [options]

Options:
 -h, --help                    Print this page
 -c                            Configuration file (default /source/falco/falco.yaml, /etc/falco/falco.yaml)
 -A                            Monitor all events, including those with EF_DROP_SIMPLE_CONS flag.
 --alternate-lua-dir <path>    Specify an alternate path for loading Falco lua files
 -b, --print-base64            Print data buffers in base64.
                               This is useful for encoding binary data that needs to be used over media designed to.
 --cri <path>                  Path to CRI socket for container metadata.
                               Use the specified socket to fetch data from a CRI-compatible runtime.
 -d, --daemon                  Run as a daemon.
 --disable-cri-async           Disable asynchronous CRI metadata fetching.
                               This is useful to let the input event wait for the container metadata fetch
                               to finish before moving forward. Async fetching, in some environments leads
                               to empty fields for container metadata when the fetch is not fast enough to be
                               completed asynchronously. This can have a performance penalty on your environment
                               depending on the number of containers and the frequency at which they are created/started/stopped
 --disable-source <event_source>
                               Disable a specific event source.
                               Available event sources are: syscall, k8s_audit.
                               It can be passed multiple times.
                               Can not disable both the event sources.
 -D <substring>                Disable any rules with names having the substring <substring>. Can be specified multiple times.
                               Can not be specified with -t.
 -e <events_file>              Read the events from <events_file> (in .scap format for sinsp events, or jsonl for
                               k8s audit events) instead of tapping into live.
 -k <url>, --k8s-api <url>
                               Enable Kubernetes support by connecting to the API server specified as argument.
                               E.g. "http://admin:password@127.0.0.1:8080".
                               The API server can also be specified via the environment variable FALCO_K8S_API.
 -K <bt_file> | <cert_file>:<key_file[#password]>[:<ca_cert_file>], --k8s-api-cert <bt_file> | <cert_file>:<key_file[#password]>[:<ca_cert_file>]
                               Use the provided files names to authenticate user and (optionally) verify the K8S API server identity.
                               Each entry must specify full (absolute, or relative to the current directory) path to the respective file.
                               Private key password is optional (needed only if key is password protected).
                               CA certificate is optional. For all files, only PEM file format is supported. 
                               Specifying CA certificate only is obsoleted - when single entry is provided 
                               for this option, it will be interpreted as the name of a file containing bearer token.
                               Note that the format of this command-line option prohibits use of files whose names contain
                               ':' or '#' characters in the file name.
 -L                            Show the name and description of all rules and exit.
 -l <rule>                     Show the name and description of the rule with name <rule> and exit.
 --list [<source>]             List all defined fields. If <source> is provided, only list those fields for
                               the source <source>. Current values for <source> are "syscall", "k8s_audit"
 -m <url[,marathon_url]>, --mesos-api <url[,marathon_url]>
                               Enable Mesos support by connecting to the API server
                               specified as argument. E.g. "http://admin:password@127.0.0.1:5050".
                               Marathon url is optional and defaults to Mesos address, port 8080.
                               The API servers can also be specified via the environment variable FALCO_MESOS_API.
 -M <num_seconds>              Stop collecting after <num_seconds> reached.
 -N                            When used with --list, only print field names.
 -o, --option <key>=<val>      Set the value of option <key> to <val>. Overrides values in configuration file.
                               <key> can be a two-part <key>.<subkey>
 -p <output_format>, --print <output_format>
                               Add additional information to each falco notification's output.
                               With -pc or -pcontainer will use a container-friendly format.
                               With -pk or -pkubernetes will use a kubernetes-friendly format.
                               With -pm or -pmesos will use a mesos-friendly format.
                               Additionally, specifying -pc/-pk/-pm will change the interpretation
                               of %container.info in rule output fields.
 -P, --pidfile <pid_file>      When run as a daemon, write pid to specified file
 -r <rules_file>               Rules file/directory (defaults to value set in configuration file, or /etc/falco_rules.yaml).
                               Can be specified multiple times to read from multiple files/directories.
 -s <stats_file>               If specified, append statistics related to Falco's reading/processing of events
                               to this file (only useful in live mode).
 --stats-interval <msec>       When using -s <stats_file>, write statistics every <msec> ms.
                               This uses signals, so don't recommend intervals below 200 ms.
                               Defaults to 5000 (5 seconds).
 -S <len>, --snaplen <len>
                               Capture the first <len> bytes of each I/O buffer.
                               By default, the first 80 bytes are captured. Use this
                               option with caution, it can generate huge trace files.
 --support                     Print support information including version, rules files used, etc. and exit.
 -T <tag>                      Disable any rules with a tag=<tag>. Can be specified multiple times.
                               Can not be specified with -t.
 -t <tag>                      Only run those rules with a tag=<tag>. Can be specified multiple times.
                               Can not be specified with -T/-D.
 -U,--unbuffered               Turn off output buffering to configured outputs.
                               This causes every single line emitted by falco to be flushed,
                               which generates higher CPU usage but is useful when piping those outputs
                               into another process or into a script.
 -u, --userspace               Parse events from userspace.
                               To be used in conjunction with the ptrace(2) based driver (pdig).
 -V, --validate <rules_file>   Read the contents of the specified rules(s) file and exit.
                               Can be specified multiple times to validate multiple files.
 -v                            Verbose output.
 --version                     Print version number.

rule 확인

먼저 falco에서 기본 내장된 rule을 확인해볼수 있다.
기본적으로 /etc/falco/falco_rules.yaml 및 /etc/falco/faclo_rules.local.yaml, /etc/falco/falco_rules.d/ 내에 있는 rule들을 loading 한다.

jacob@jacob-laptop:~/workspaces$ falco -L
Sun Jan  3 17:32:57 2021: Falco version 0.26.2 (driver version 2aa88dcf6243982697811df4c1b484bcbe9488a2)
Sun Jan  3 17:32:57 2021: Falco initialized with configuration file /etc/falco/falco.yaml
Sun Jan  3 17:32:57 2021: Loading rules from file /etc/falco/falco_rules.yaml:
Sun Jan  3 17:32:58 2021: Loading rules from file /etc/falco/falco_rules.local.yaml:
Sun Jan  3 17:32:58 2021: Loading rules from file /etc/falco/k8s_audit_rules.yaml:

Rule                                               Description
----                                               -----------
Create Symlink Over Sensitive Files                Detect symlink created over sensitive files

Create Sensitive Mount Pod                         Detect an attempt to start a pod with a volume from a
                                                   sensitive host directory (i.e. /proc). Exceptions are made
                                                   for known trusted images.


Unexpected outbound connection destination         Detect any outbound connection to a destination outside of
                                                   an allowed set of ips, networks, or domain names

Attach/Exec Pod                                    Detect any attempt to attach/exec to a pod


K8s Serviceaccount Created                         Detect any attempt to create a service account

Disallowed K8s User                                Detect any k8s operation by users outside of an allowed set
                                                   of users.

Interpreted procs inbound network activity         Any inbound network activity performed by any interpreted
                                                   program (perl, python, ruby, etc.)

container내 shell 실행 detect

간단하게 처음 소개되었던 shell에 대한 detect 과정을 알아보자.

먼저 falco를 docker daemon이 동작되는 host에서 동작시키고

jacob@jacob-laptop:~/workspaces$ sudo falco
Sun Jan  3 17:11:02 2021: Falco version 0.26.2 (driver version 2aa88dcf6243982697811df4c1b484bcbe9488a2)
Sun Jan  3 17:11:02 2021: Falco initialized with configuration file /etc/falco/falco.yaml
Sun Jan  3 17:11:02 2021: Loading rules from file /etc/falco/falco_rules.yaml:
Sun Jan  3 17:11:02 2021: Loading rules from file /etc/falco/falco_rules.local.yaml:
Sun Jan  3 17:11:02 2021: Loading rules from file /etc/falco/k8s_audit_rules.yaml:
Sun Jan  3 17:11:03 2021: Starting internal webserver, listening on port 8765

새로운 terminal 을 열어 아래와 같이 test container를 실행해보자

jacob@jacob-laptop:~/workspaces/falco$ docker run --name pyweb -d python:3.8-alpine python -m http.server
Unable to find image 'python:3.8-alpine' locally
3.8-alpine: Pulling from library/python
801bfaa63ef2: Pull complete 
7678dd7631a2: Pull complete 
4c6139ab40d8: Pull complete 
ff5ef8cd8062: Pull complete 
73dee1f700a1: Pull complete 
Digest: sha256:60a24db20ad0121b3440681ded50a75b762ceb9cb0855847a4b25d291e9de8c2
Status: Downloaded newer image for python:3.8-alpine
37107f2393681025fce6cc869c454a9b2a4ea3ffff31c1ea7bb97df6ae43aa01
jacob@jacob-laptop:~/workspaces/falco$ docker ps -a
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
37107f239368        python:3.8-alpine   "python -m http.serv…"   4 seconds ago       Up 3 seconds                            pyweb

위와 같이 pyweb container가 동작됨을 확인할 수 있다.
이제 해당 container에 shell로 접속해보자

jacob@jacob-laptop:~/workspaces/falco$ docker exec -it pyweb sh
/ #

위와 같이 접속시 falco는 다음과 같은 log를 출력시켰다.

17:19:19.021296662: Notice A shell was spawned in a container with an attached terminal (user=root user_loginuid=-1 pyweb (id=37107f239368) shell=sh parent=runc cmdline=sh terminal=34816 container_id=37107f239368 image=python)

즉, sh shell로 pyweb에 login하였다라는 로그가 출력되어진다.

민감한 디렉토리 mount detect

이번에는 민감한 디렉토리인 /proc을 mount시켜보자.

jacob@jacob-laptop:~/workspaces/falco$ docker run --name pyweb -v /proc/:/proc -d python:3.8-alpine python -m http.server
463979921fe7fcf6948ab82a6b31d2be6da02c1d11c5e5b5984927d8470c82ac

위 명령을 통해 민감한 디렉토리인 /proc이 mount 됨에 따라 다음과 같은 메세지를 출력한다.

17:25:33.807258097: Notice Container with sensitive mount started (user=root user_loginuid=0 command=container:463979921fe7 pyweb (id=463979921fe7) image=python:3.8-alpine mounts=/proc:/proc::true:rprivate)
17:25:34.370200014: Notice Container with sensitive mount started (user=root user_loginuid=-1 command=python -m http.server pyweb (id=463979921fe7) image=python:3.8-alpine mounts=/proc:/proc::true:rprivate)

관련된 설정은 다음과 같다. (falco_rules.yaml에 기본적인 rule이 존재한다.)

jacob@jacob-laptop:~/workspaces/falco$ grep -A10 "macro: sensitive_mount" /etc/falco/falco_rules.yaml 
- macro: sensitive_mount
  condition: (container.mount.dest[/proc*] != "N/A" or
              container.mount.dest[/var/run/docker.sock] != "N/A" or
              container.mount.dest[/var/run/crio/crio.sock] != "N/A" or
              container.mount.dest[/var/lib/kubelet] != "N/A" or
              container.mount.dest[/var/lib/kubelet/pki] != "N/A" or
              container.mount.dest[/] != "N/A" or
              container.mount.dest[/home/admin] != "N/A" or
              container.mount.dest[/etc] != "N/A" or
              container.mount.dest[/etc/kubernetes] != "N/A" or
              container.mount.dest[/etc/kubernetes/manifests] != "N/A" or

helm chart를 사용한 falco 활용

helm chart를 활용해 각 node에 falco pod가 동작되며 custom kernel module인 falco.ko도 자동으로 load 시키는 과정이다.
먼저 helm chart repo를 추가한다.

helm repo add falcosecurity https://falcosecurity.github.io/charts

추가한 falco repo에서 helm chart를 아래와 같이 설치한다.

[root@jacobbaek-deploy centos]# helm install falco falcosecurity/falco
NAME: falco
LAST DEPLOYED: Sun Jan  3 06:07:35 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Falco agents are spinning up on each node in your cluster. After a few
seconds, they are going to start monitoring your containers looking for
security issues.
No further action should be required.

각 노드에 daemonset으로 pod가 동작되게 된다.

[root@jacobbaek-deploy falco]# kubectl get all
NAME              READY   STATUS    RESTARTS   AGE
pod/falco-45vwp   1/1     Running   0          73m
pod/falco-6x45l   1/1     Running   0          73m
pod/falco-9vnbx   1/1     Running   0          73m
pod/falco-rs6xl   1/1     Running   0          73m
pod/falco-w569d   1/1     Running   0          73m

NAME                   DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
daemonset.apps/falco   5         5         5       5            5           <none>          73m

이제 각 노드에서 동작중인 falco pod에 log로 확인시 위에서 알아보았던 detect되는 내용들이 출력되게 된다.

참고
falco로 detect 되는 event들을 prometheus에서 metic으로 활용할수도 있다.

개인적인 총평

Falco homepage에서는 다음과 같은 소개를 하고 있다.
"Falco lets you continuously monitor and detect container, application, host, and network activity"
즉 container외에도 host, application도 가능하다는 뜻이다.

사실 Kubernetes에 incubating project로 되어 있기에 kubernetes에 종속적인 기능들이 다 일거라 생각했으나
실제 내용은 kubernetes를 넘어선 다양한 환경에서 사용이 가능하여 보안적 측면에서 고려해볼만한 도구라 판단된다.

참고사이트

감사하게도 한글 사이트를 제공하고 있다.
(단, 그냥 번역기를 사용한 수준인듯하다..)

'Cloud > Cloud Native' 카테고리의 다른 글

Falco  (0) 2021.01.03
Rook  (0) 2020.12.14
Hashicorp Waypoint  (0) 2020.10.27
How to use Hashicorp Waypoint  (0) 2020.10.27
Metallb on Minikube  (0) 2020.10.27
starboard  (0) 2020.10.03
댓글
댓글쓰기 폼