关注互联网应用及运维技术的个人博客

Flask gevent 多进程WSGI

python多进程gevent,Flask gevent multiprocess WSGI,Flask gevent 多进程WSGI,Flask多进程gevent。

题外话:Flask,Instagram据说扛住了上亿日活,以前是Django。其是一个非常优秀的web service 框架,简洁灵活,可以利用大量的第三方组件和模块来快速开发。

如今,Instagram 的总注册用户达到 30 亿,月活用户超过 7 亿 (作为对比,微信最新披露的月活跃用户为 9.38 亿)。而令人吃惊的是,这么高的访问量背后,竟完全是由以速度“慢”著称的 Python + Django 支撑。

时至今日,即使已经拥有超过 30 亿的注册用户。Instagram 仍然是 Python 和 Django 的重度使用者。Instagram 的工程师 Hui Ding 说到: 『一直到用户 ID 已经超过了 32bit int 的限额(约为 20 亿),Django 本身仍然没有成为我们的瓶颈所在。』

 flask + gevent + multiprocess + wsgi简介

常常大家都是用gunicorn来解决flask后端部署并发的问题, 然而觉得自启多进程是为更优雅的高并发方式。这样就不需要gunicorn了。也没有额外的第三方部署工作,于是有了以下flask + gevent + multiprocess + wsgi的测试

lask + gevent + multiprocess + wsgi程序代码ca.py

# coding: utf-8
# flask + gevent + multiprocess + wsgi

from gevent import monkey
from gevent.pywsgi import WSGIServer
monkey.patch_all()

import datetime
import os
from multiprocessing import cpu_count, Process
from flask import Flask, jsonify


app = Flask(__name__)

@app.route("/ca", methods=['GET'])
def function_benchmark():
    return jsonify(
        {
            "status": "ok",
            "time": datetime.datetime.now().strftime('%Y-%m-%d %H:%M'),
            "pid": os.getpid()
        }
    ), 200

def run(MULTI_PROCESS):
    if MULTI_PROCESS == False:
        WSGIServer(('0.0.0.0', 8080), app).serve_forever()
    else:
        mulserver = WSGIServer(('0.0.0.0', 8080), app)
        mulserver.start()

        def server_forever():
            mulserver.start_accepting()
            mulserver._stop_event.wait()

        for i in range(cpu_count()):
            p = Process(target=server_forever)
            p.start()

if __name__ == "__main__":
    # 单进程 + 协程
    run(False)
    # 多进程 + 协程
    # run(True)

测试环境

机器配置

  • 4vCPU
  • 8GB

环境配置

  • Centos 8
  • Python 3.6.8
  • Flask  1.1.2
  • Gevent 1.5.0

压测工具

  • WRK
  • 4H8G机器

单进程 + 协程 + WSGI

测试结果

QPS : 2361 r/s

该模式启动了一个进程

[root@vm5 ~]# nohup python3 ca.py &
[1] 9371
[root@vm5 ~]# nohup: ignoring input and appending output to 'nohup.out'

[root@vm5 ~]# ps -ef | grep ca
root      9371  4184  4 03:33 pts/0    00:00:00 python3 ca.py
root      9377  4184  0 03:33 pts/0    00:00:00 grep --color=auto ca
[root@vm5 ~]#

测试详情

[root@vm4 wrk]# curl 10.10.10.5:8080/ca
{"pid":9371,"status":"ok","time":"2020-04-20 04:19"}
[root@vm4 wrk]# curl 10.10.10.5:8080/ca
{"pid":9371,"status":"ok","time":"2020-04-20 04:19"}
[root@vm4 wrk]# curl 10.10.10.5:8080/ca
{"pid":9371,"status":"ok","time":"2020-04-20 04:19"}
[root@vm4 wrk]# wrk -t12 -c400 -d30s http://10.10.10.5:8080/ca
Running 30s test @ http://10.10.10.5:8080/ca
  12 threads and 400 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   164.40ms   21.15ms 515.30ms   80.85%
    Req/Sec   199.61     47.81   565.00     70.38%
  71237 requests in 30.10s, 10.95MB read
Requests/sec:   2366.72
Transfer/sec:    372.44KB
[root@vm4 wrk]# wrk -t20 -c800 -d30s http://10.10.10.5:8080/ca   
Running 30s test @ http://10.10.10.5:8080/ca
  20 threads and 800 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   329.39ms   93.86ms   1.78s    92.80%
    Req/Sec   126.19     80.69   696.00     67.71%
  71075 requests in 30.10s, 10.92MB read
  Socket errors: connect 0, read 0, write 0, timeout 97
Requests/sec:   2361.39
Transfer/sec:    371.65KB
[root@vm4 wrk]#

多进程 + 协程 + WSGI

测试结果

QPS : 7500 r/s

该模式启动了4+1个进程(4vCPU)

[root@vm5 ~]# nohup python3 ca.py &
[1] 9537
[root@vm5 ~]# nohup: ignoring input and appending output to ‘nohup.out’

[root@vm5 ~]# ps -ef | grep ca
root 9537 4184 5 04:32 pts/0 00:00:00 python3 ca.py
root 9542 9537 0 04:32 pts/0 00:00:00 python3 ca.py
root 9543 9537 0 04:32 pts/0 00:00:00 python3 ca.py
root 9544 9537 0 04:32 pts/0 00:00:00 python3 ca.py
root 9545 9537 0 04:32 pts/0 00:00:00 python3 ca.py
root 9547 4184 0 04:32 pts/0 00:00:00 grep –color=auto ca
[root@vm5 ~]#

测试详情

[root@vm4 wrk]# curl 10.10.10.5:8080/ca
{“pid”:9543,”status”:”ok”,”time”:”2020-04-20 04:34″}
[root@vm4 wrk]# curl 10.10.10.5:8080/ca
{“pid”:9542,”status”:”ok”,”time”:”2020-04-20 04:34″}
[root@vm4 wrk]# curl 10.10.10.5:8080/ca
{“pid”:9545,”status”:”ok”,”time”:”2020-04-20 04:34″}
[root@vm4 wrk]# wrk -t12 -c400 -d30s http://10.10.10.5:8080/ca
Running 30s test @ http://10.10.10.5:8080/ca
12 threads and 400 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 56.10ms 15.16ms 187.30ms 85.05%
Req/Sec 590.77 79.95 830.00 67.97%
212138 requests in 30.08s, 32.60MB read
Requests/sec: 7051.89
Transfer/sec: 1.08MB
[root@vm4 wrk]# wrk -t20 -c800 -d30s http://10.10.10.5:8080/ca
Running 30s test @ http://10.10.10.5:8080/ca
20 threads and 800 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 101.59ms 40.23ms 337.80ms 66.06%
Req/Sec 394.20 109.48 0.97k 74.47%
235844 requests in 30.10s, 36.25MB read
Requests/sec: 7835.77
Transfer/sec: 1.20MB
[root@vm4 wrk]#

赞(0)
未经允许不得转载:飞天狒狒 » Flask gevent 多进程WSGI

评论 抢沙发

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址