程序员人生 网站导航

Keystone 高可靠性部署与性能测试

栏目:服务器时间:2014-12-20 08:37:38

Goal

   

     Keystone Region 为跨地域的 Openstack 集群提供了统1的认证和用户租户管理。目前公司在国内外部署了数10套 Openstack 集群,其中既有集群在内网,又有集群在公网;既有 Havana 集群,也有 Icehouse 集群;既有 nova-network 集群,又有 Neutron 集群,以下图:



    为了集中管理,全局同享1个 Keystone Server, 因此对 Keystone Server 的安全性和性能,都有特殊的要求。



    安全性通过 SSL 实现和避免 DDOS 实现,可靠性通过 Apache、Haproxy、mysqlcluster 实现(关于 openstack 整体 HA 的实现,可以参考  http://blog.csdn.net/wsfdl/article/details/41386155),以下图:


 

Deployment

物理主机信息

Host Name            IP                         VIP/DNS                                 CPU                                        Memory
keystone01            internal_ip01        public_ip/keystone-server       E5⑵620(24 Processor)          64G
keystone02            internal_ip02        public_ip/keystone-server       E5⑵620(24 Processor)          64G


说明:若无注明,keystone01 和 keystone02 的部署与配置相同

# yum   -y   install   mysql mysql-server MySQL-python
# yum   -y   install   openstack-keystone python-keystoneclient
# yum   -y   install   haproxy
# yum   -y   install   httpd
# yum   -y   install   keepalived
# yum   -y   install   haproxy
# yum   -y   install   httpd
# yum   -y   install   keepalived

Configuration

/etc/keystone/keystone.conf
[DEFAULT]
public_endpoint=https://keystone-server/main/
admin_endpoint=https://keystone-server/admin/
[database]
connection=mysql://keystone:keystonepass@mysqlserver/keystone
max_pool_size=500
[signing]
token_format=UUID
[ssl]
cert_subject=/C=US/ST=Unset/L=Unset/O=Unset/CN=keystone-server
[token]
provider=keystone.token.providers.uuid.Provider



/etc/httpd/conf.d/wsgi-keystone.conf
NameVirtualHost *:5000
Listen internal_ip0x:5000
<VirtualHost *:5000>
ServerName keystone-main
WSGIScriptAlias /main  /var/www/cgi-bin/keystone/main
ErrorLog /var/log/keystone/apache2-main-error.log
LogLevel debug
CustomLog /var/log/keystone/apache2-main-access.log common
</VirtualHost>
 
NameVirtualHost *:35357
Listen internal_ip0x:35357
<VirtualHost *:35357>
ServerName keystone-admin
WSGIScriptAlias /admin  /var/www/cgi-bin/keystone/admin
ErrorLog /var/log/keystone/apache2-admin-error.log
LogLevel debug
CustomLog /var/log/keystone/apache2-admin-access.log common
</VirtualHost>



/etc/haproxy/haproxy.cfg
global
    daemon
    log 127.0.0.1 local3
 
defaults
    maxconn 4000
    log     global
    timeout server 10s
    timeout connect 10s
    timeout client 10s
    mode http
    option forwardfor
    option http-server-close
    log global
 
listen stats
    mode http
    bind public_ip:8000
    stats enable
    stats hide-version
    stats uri     /
    stats realm   Haproxy Statistics
    stats auth    lecloud:openstack
    stats admin if TRUE
 
frontend keystone_frontend
    bind public_ip:443 ssl crt /etc/haproxy/keystone_https.pem
    reqadd X-Forwarded-Proto: https
    acl admin_path path_beg  /admin
    acl main_path  path_beg  /main
    use_backend admin_backend if admin_path
    use_backend main_backend if main_path
 
backend admin_backend
    balance roundrobin
    redirect scheme https if !{ ssl_fc }
    server keystone-server-01 internal_ip01:35357 check inter 10s
    server keystone-server-02 internal_ip02:35357 check inter 10s
 
backend main_backend
    balance roundrobin
    redirect scheme https if !{ ssl_fc }
    server keystone-server-01 internal_ip01:5000 check inter 10s
    server keystone-server-02 internal_ip02:5000 check inter 10s

/etc/keepalived/keepalived.conf
vrrp_script haproxy-check {
    script "killall -0 haproxy"
    interval 2
    weight 10
}
 
vrrp_instance openstack-vip {
    state MASTER               # 注:keystone01 为 MASTER, keystone02 为 BACKUP
    priority 102
    interface eth0
    virtual_router_id 108
    advert_int 3
    virtual_ipaddress {
        public_ip
    }
    track_script {
        haproxy-check
    }
}


# mkdir   /var/www/cgi-bin/keystone/
# cp   /usr/share/keystone/keystone.wsgi   /var/www/cgi-bin/keystone/
# ln   -s   /var/www/cgi-bin/keystone/keystone.wsgi   /var/www/cgi-bin/keystone/admin
# ln   -s   /var/www/cgi-bin/keystone/keystone.wsgi   /var/www/cgi-bin/keystone/main
# service   httpd   start
# chkconfig   httpd   on

# keystone-manage   ssl_setup   --keystone-user   keystone   --keystone-group   keystone                     注:keystone01
# cat  /etc/keystone/ssl/certs/keystone.pem   /etc/keystone/ssl/private/keystonekey.pem   >   /etc/haproxy/keystone_https.pem              
注:keystone01,同时把 keystone_https.pem 拷贝至 keystone02 /etc/haproxy/ 目录下

# (crontab   -l   -u   keystone   2>&1 | grep   -q   token_flush)   ||   echo '@dayly   /usr/bin/keystone-manage   token_flush >/var/log/keystone/keystone-tokenflush.log   2>&1'   >>   /var/spool/cron/keystone
# echo   "net.ipv4.ip_nonlocal_bind = 1"   >>   /etc/sysctl.conf

# sysctl   -p
# service   haproxy   start
# chkconfig   haproxy   on
# service   keepalived   start
# chkconfig   keepalived   on


Benchmark


Configure Rally

关于 Rally,详情请参见  Openstack 性能测试 http://blog.csdn.net/wsfdl/article/details/41654373

# git   clone   https://git.openstack.org/stackforge/rally   &&   cd   rally
# ./rally/install_rally.sh   -v
source   /opt/rally/bin/activate
#  rally   deployment   create   --filename=existing.json   --name=existing
#  rally   -v   task   start   create-user.json

(rally)[root@controller rally]# cat existing.json {     "type": "ExistingCloud",     "auth_url": "https://keystone-server/admin/v2.0",     "admin": {         "username": "test",         "password": "test",         "tenant_name": "test"     } }
create-user.json
{     "KeystoneBasic.create_user": [         {             "args": {                 "name_length": 10             },             "runner": {                 "type": "constant",                 "times": 10000,                 "concurrency": 900             }         }     ] } 

Result

注:以创建用户为例,1个并发数(Concurrency),包括两个 HTTPS 要求(1个为申请 token,另外一个为创建用户)。此处仅给出 mysql(单点) 数据库下keystone server 的并发性能。







------分隔线----------------------------
------分隔线----------------------------

最新技术推荐