General answer: Caching is mandatory for performance.

• Particular answer to proxy cache:
  • I've configured nginx proxycache on /dev/shm for performance (on ram cache). 1 minute for proxycache should be fine if the content is changing constantly, cause nginx will “rebuild/compile” the cache after 1 minute but inside that minute, it will use the already compiled version of the file.
• Particular answer to static content:
  • Static content to 1 hour maybe is aggressive, normally I set it up to 1day or more, also depends on the type of application.
• Particular answer to proxied content:
  • I Think we must include a header like “If-Modified-Since” to invalidate local cache if the file has been changed on the server.

I did some mistakes, so I create some versions of it:

• name : dodgermarfeeltest_003
• id : ami-0bc1be25784321fc1

I've configured nginx with a bare minimal setup:

/etc/nginx/nginx.conf

user www-data;
worker_processes auto;
pid /run/nginx.pid;
events {
        worker_connections 768;
}
http {
        server_tokens off;
        sendfile on;
        tcp_nopush on;
        tcp_nodelay on;
 
        ## Start: Timeouts ##
        client_body_timeout   10;
        client_header_timeout 10;
        keepalive_timeout     5 5;
        send_timeout          10;
        ## End: Timeouts ##
 
        types_hash_max_size 2048;
        include /etc/nginx/mime.types;
        default_type application/octet-stream;
        ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
        ssl_prefer_server_ciphers on;
        access_log /var/log/nginx/access.log;
        error_log /var/log/nginx/error.log;
 
        # compress everything
        gzip on;
        # disabled by marfeel request:
        #       Enable gzip for all the requests (proxied request included)
        #       you have chosen it :-)
        #gzip_disable "msie6";
 
        #include /etc/nginx/conf.d/*.conf;
        include /etc/nginx/conf.d/proxy_cache.conf;
        include /etc/nginx/sites-enabled/*;
}

I've setup a .service for systemd named marfeeltest which is inside the AMI: <file yaml /etc/systemd/system/marfeel_test.service> [Unit] Description=marfeel test service After=auditd.service systemd-user-sessions.service time-sync.target [Service] User=marfeel TimeoutStartSec=0 Type=simple KillMode=control-group WorkingDirectory=/opt/test/Marfeel-appserverpythontestapp-2937d4f8673c ExecStart=/usr/bin/python3 -m http.server –cgi 8080 Restart=no [Install] WantedBy=multi-user.target </file> ===== Which code have you added to the user-data on launching the instance? ===== <code yaml> #cloud-config runcmd: - [ mkdir, -p, /opt/test ] - [ wget, -O/opt/test/master.tar.gz, “https://bitbucket.org/Marfeel/appserverpythontestapp/get/master.tar.gz” ] - [ tar, xzfv, /opt/test/master.tar.gz, -C, /opt/test/ ] - [ systemctl, start, marfeel_test ] - [ systemctl, restart, nginx ] </code> ===== Security group ===== I create the 001 group with https, then I remove it in the 2nd version: * name: dodgerlaunch_002

* name: ''marfeel_scalinggroup_001''

Done as required with a very simple bash script.
Dependencies for running the script:

• aws-cli
• jq

This script should be run by root.

Code:

make_it_cool.sh

#!/bin/bash
 
# Exit codes:
#  1 :
#  2 :
#  3 :
#  4 :
 
 
########################################################################
# INIT
########################################################################
CONFIGFILE="$(dirname $(readlink -f $0))/$(basename $(readlink -f $0) .sh).config"
 
########################################################################
#/INIT
########################################################################
 
########################################################################
#
# CONSTANTS
#
########################################################################
 
# colors
BOLD="\e[1m"
GREEN="\e[32m"
LIGHTGREEN="${BOLD}${GREEN}"
RED="\033[1;31m"
LIGHTRED="\033[1;31m"
BLUE="\e[34m"
LIGHTBLUE="${BOLD}${BLUE}"
YELLOW="\e[33m"
LIGHTYELLOW="${BOLD}${YELLOW}"
WHITE="\033[0;37m"
RESET="\033[0;00m"
 
NOW="$(date +%Y%m%d%H%M%S)"
 
 
########################################################################
#
# / CONSTANTS
#
########################################################################
 
 
 
 
########################################################################
#
# VARIABLES
#
########################################################################
 
 
SCRIPTLOG="$(dirname `readlink -f $0`)/logs/$(basename $0 .sh)_script_${NOW}.log"
SCRIPTLOGERR="$(dirname `readlink -f $0`)/logs/$(basename $0 .sh)_script_${NOW}.err"
 
TMPFILE=$(mktemp)
HADYNAMICCFG=$(mktemp)
 
########################################################################
#
# / VARIABLES
#
########################################################################
 
 
########################################################################                                                                                                                                [49/5875]
#                                                                                                                                                                                                                
# FUNCTIONS
#
########################################################################
 
 
usage()
{
    printf "%s${LIGHTRED}USAGE:${RESET}
    $0
 
	Read documentation here:
	https://wiki.ciberterminal.net/doku.php?id=marfeel:test#dynamic_set_up"
    # VERY INITIAL CHECKS
}
 
printmsg()
{
   echo -e "$*"
}
 
output_log()
{
    if [[ "${QUIETOUTPUT}" == true ]]; then
        printmsg "$*" >> ${OUTPUTFILE}
    else
        printmsg "$*" | tee -a "${OUTPUTFILE}"
    fi
}
 
abort_message()
{
    printmsg "ERROR: $*"
    exit 1
}
 
# debug_me uses variable ${DEBUGME}
debug_me()
{
    if [[ "${DEBUGME}" && ${DEBUGME} -eq 0 ]] ; then 
        echo -e "${LIGHTBLUE}DEBUG: ${RESET}$*"
    fi
}
 
 
########################################################################
#
# / FUNCTIONS
#
########################################################################
 
########################################################################
#
# MAIN
#
########################################################################
 
[[ ! -d $(dirname ${SCRIPTLOG}) ]] && mkdir -p $(dirname ${SCRIPTLOG})
 
# DETECTING if the script is run by cron
if [[ "$(tty)" = "not a tty" ]] ; then
	set -x
	exec > ${SCRIPTLOG}
	exec 2> ${SCRIPTLOGERR}
fi
 
if [[ ${DEBUG} -eq 0 ]] ; then
    echo -e "${BLUE}DEBUGMODE${RESET} is on"
    echo -e "\t SCRIPTLOG will be ${SCRIPTLOG}"
    echo -e "\t SCRIPTLOGERR will be ${SCRIPTLOGERR}"
    set -x
    exec 2> ${SCRIPTLOGERR}
fi
 
 
[[ ! -f ${CONFIGFILE} ]] && echo -e "${LIGHTRED} CONFIGFILE ${CONFIGFILE} NOT FOUND${RESET}" && exit 1
. ${CONFIGFILE}
 
[[ ${DEBUG} -eq 0 ]] && DEBUGME="bash -x"
 
${AWSCLI} ec2 describe-instances --filters Name=instance-type,Values=t2.nano Name=instance-state-name,Values=running > ${TMPFILE}
let x=0
for PRIVATEIP in $(cat ${TMPFILE} | ${JQ} '.Reservations[] | .Instances[] | "\(.PrivateIpAddress) \(.PublicIpAddress) \(.Tags)"' | egrep "${SCALINGTAG}" | awk 'BEGIN{FS="[ \"]"}{print $2}') ; do
	echo -e "\tserver marfeel_nginx_${x} ${PRIVATEIP}:80 check maxconn 512" >> ${HADYNAMICCFG}
	let x++
done
 
if [[ $(cat ${HADYNAMICCFG} | wc -l) -ge ${MINBACKENDS} ]] ; then
	# ok, overwriting config
	echo -e "#### WARNING THIS CONFIG WILL BE REWRITTEN BY CRONJOB" > ${HAPROXYCONFIG}
	cat ${HAPROXYTEMPLATE} >> ${HAPROXYCONFIG}
	cat ${HADYNAMICCFG} >> ${HAPROXYCONFIG}
	echo -e "#### WARNING THIS CONFIG WILL BE REWRITTEN BY CRONJOB" >> ${HAPROXYCONFIG}
	systemctl reload haproxy
fi
 
rm -f ${TMPFILE}
rm -f ${HADYNAMICCFG}
 
exit ${EXITCODE}
 
########################################################################
#
# / MAIN
#
########################################################################

Config file:

make_it_cool.config

# Our template for re-generate the config
HAPROXYTEMPLATE=/etc/haproxy/haproxy.cfg.TMPL
# The configfile itself
HAPROXYCONFIG=/etc/haproxy/haproxy.cfg
# Minimum amount of backends that must be running, less than this, the config file won't be changed
MINBACKENDS=2
 
# Tag for the scaling group (backend servers will be filtered by this tag)
SCALINGTAG="marfeel_scalinggroup_001"
 
# generic setup
AWSCLI=/usr/bin/aws
JQ=/usr/bin/jq
 
# SET to 0 for DEBUG
DEBUG=0

Setup crontab with desired frequency for refresh, for example:

*/3 *   * * *   root    /root/marfeel_auto_updater/make_it_cool.sh