k8s service: how to create ELB in a private subnet

For a k8s service in a AWS private subnet. By default it won’t create ELB. In order to get k8s doing all the grunt work of ELB creation, register k8s worker instance with the ELB, add listener, tag it property. there is a magic annotation here.

magic annotation

kubectl annotate the service

kubectl --namespace=guestbook annotate service frontend  service.beta.kubernetes.io/aws-load-balancer-internal=0.0.0.0/0

Advertisements
Posted in k8s | Tagged | Leave a comment

linux file security bits

Posted in linux, Uncategorized | Tagged | Leave a comment

How to expose docker app on LAN

docker-machine create –driver virtualbox webexposed

docker-machine stop webexposed

VBoxManage modifyvm webexposed –natdnshostresolver1 on

VBoxManage modifyvm webexposed –natpf1 ‘http,tcp,,8080,,80’

docker-machine start webexposed

docker-machine create –driver virtualbox webexposed

docker-compose up

browser to http://lan_ip:8080/

Posted in Uncategorized | Leave a comment

VBoxManage & Docker

 

Popular VBoxManage cmd

Vagrant

  • vagrant package –base   (this creates a new box)
  • vagrant box add docker-centos package.box
Posted in Uncategorized | Leave a comment

gpg cheatsheet

  • install pgpdump util for viewing the PGP keys.
  • import PGP keys into a custom private key ring:  The public and private are both in private.pgp
    • gpg –import –no-default-keyring –keyring /tmp/public –secret-keyring /tmp/private private.pgp
  • list secret keys in a non default ring:
    • gpg -K –secret-keyring /tmp/private
  • list  keys in default public and secret rings:
    • gpg -K, gpg -k

A script

# import the public and private key pair prior to the encryption

gpg –import –no-default-keyring –secret-keyring ${SECRET_KEYRING} ${PRIVATE_KEY}

chmod 400 ${SECRET_KEYRING} ${PRIVATE_KEY} ${PASSPHRASE}

# decrypt
cd ${input_dir}

for pgp_file in *.gpg
do
unenc_file=${pgp_file%.gpg}
gpg –secret-keyring ${SECRET_KEYRING} –batch –output ${output_dir}/${unenc_file} –passphrase-fd 0 $pgp_file < ${PASSPHRASE}
done

Posted in gpg | Leave a comment

sensu

Sensu uses the embedded ruby

  • /etc/default/sensu
    EMBEDDED_RUBY=true
  • /opt/sensu/embedded/bin/gem list
  • /opt/sensu/embedded/bin/gem install sensu-plugin

mailer.rb handler set up

  • /etc/sensu/handlers/mailer.rb downloaded from sensu-community-plugin.
  • /etc/sensu/conf.d/mailer.json for configuration of mailer.rb
  • Use this handler “mailer” in a check such as /etc/sensu/conf.d/check_apache.json
  • {
    “checks”: {
    “apache_check”: {
    “handlers”: [“mailer”],
    “command”: “/etc/sensu/plugins/check-apache.rb”,
    “interval”: 60,
    “subscribers”: [ “web” ]
    }
    }
    }
Posted in sensu | Tagged | Leave a comment

high performance python tips

Quoted from http://ict.swisscom.ch/2015/02/pyperformance/

  •  I/O bound problems can make good use of multi-threading (where the GIL is released during I/O) or asynchronous programming.
  • CPU-bound problems can be addressed by better algorithms (nothing beats an algorithm with less computational complexity), using array-based programming (NumPy), using various problem-specific packages written in a compiled language, or using Cython, a mix of C and Python.
  • Application-level caches are also helpful, because no computation is always faster than the fastest possible computation.

The GIL constraint is removed when multiple processes are used, each with its own Python interpreter and GIL. This works nicely for problems that don’t require massive interaction between data or even massive amounts of read-only data.

In my own work with Quantax, the Swisscom Market Risk System, which is written in Python, we always face demand for increased speed. Using a lot of NumPy and many levels of application caches, we achieve about 25000 valuations of financial instruments per second on one core of a laptop CPU.

However, the price for this is complexity of cache invalidations, and complicated code to map the problem to NumPy.

We use processes at a relative coarse-grained level, as worker processes to calculate reports. The main issue of processes is the massive amount of common data the financial calculations require, leading to rather large memory consumption per process. However, there is rarely more than one logical process that modifies the objects (by changing transactions or rates).

Posted in python | Tagged | Leave a comment