JMeter可以通过os命令调用Python脚本,Python同样可以通过系统命令调用JMeter执行压测
Python调用JMeter
-
首先要安装JMeter,官方下载地址
解压并配置配置环境路径或建立软连,使得在命令输入jmeter便可以执行,如
Copyunzip apache-jmeter-5.3.zip
mv apache-jmeter-5.3 /usr/loca/jmeter
ln -s /usr/local/jmeter/bin/jmeter /usr/bin/jmeter
ln -s /usr/local/jmeter/bin/jmeter-server /usr/bin/jmeter-server
打开JMeter并设计一个测试计划保存为testplan.jmx
使用Python调用JMeter压测并生成报告
Python中可以使用os.system()或supprocess.Popen()调用系统命令,前者实时显示在屏幕上,后者可以获取到屏幕输出信息。
使用Python调用JMeter运行及生成报告的命令如下。
Copyimport subprocess
jmx_file = 'testplan.jmx' # jmx文件路径
result_file = 'result.jtl' #
log_file = 'run.log'
report_dir = 'report'
run_cmd = f'jmeter -n -t {jmx_file} -l {result_file} -j {log_file}' # 无界面运行JMeter压测命令
report_cmd = f'jmeter -g {result_file} -o {report_dir}' # 生成HTML报告命令
# 不需要获取屏幕输出是,可以使用os.system()
# os.system(run_cmd)
# os.system(report_cmd)
# 需要获取屏幕输出是,可以使用subprocess.Popen()
p1 = subprocess.Popen(run_cmd, shell=True, stdout=subprocess.PIPE)
print(p1.stdout.read().decode('utf-8'))
p2 = subprocess.Popen(report_cmd, shell=True, stdout=subprocess.PIPE)
print(p2.stdout.read().decode('utf-8'))
组装jmx
每一测试计划为一个jmx文件,jmx实际上是xml格式的,包含一些JMeter自定义的格式规范。
常用的组件有:
-
: 测试计划
-
: 线程组
-
: CSV数据文件
-
: HTTP请求
-
: HTTP请求头管理器
-
: Cookies管理器
-
: DNS缓存管理器
-
: 监听器(包括查看结果树、聚合报告等)
-
: 响应断言
-
<io.github.ningyu.jmeter.plugin.dubbo.sample.DubboSample></io.github.ningyu.jmeter.plugin.dubbo.sample.DubboSample>: 三方Dubbo请求插件
jmx中,如果一个组件有子组件,格式为
Copy<ThreadGroup 组件基本属性>
...线程组配置
</ThreadGroup><hashTree>
...内部子组件
</hashTree>
···
如果不包含子组件,则后面接一个<hashTree/> 单标签直接结束,例如:
```xml
<CSVDataSet>
... CSV数据组件配置
</CSVDataSet><hashTree/>
详细的格式可以自己使用JMeter创建一个测试计划,使用文本文件打开jmx文件查看。
使用Python组装jmx文件的方式有两种,一种是固定模板的数据渲染,一种类似JMeter的逐个组件添加,第一种比较简单。
通过分析jmx文件中的变量,我们使用jinja2模板语法,将其中的变量进行参数化,对需要判断和循环的变量设置if和for循环。
假设我们的测试计划结构为:
Copy测试计划
DNS缓存管理器
Cookies管理器
CSV文件(多个)
...
聚合报告
线程组(多个)
CSV文件(多个)
HTTP请求(或Dubbo请求)
HTTP请求头管理器
CSV文件(多个)
响应断言
察看结果树
将jmx中的关键数据抽取并组合,我使用的完整数据格式如下:
data.yaml
Copytest_plan_name:测试计划comments:测试计划描述hosts:-name:las.secoo.comaddress:112.126.120.128cookies:clear_each_iteration:'true'csv_files:- {'name':'数据文件1', 'path':'data.csv', 'varnames':'a,b', 'delimiter':','}
- {'name':'数据文件2', 'path':'data.csv', 'varnames':'c,d', 'delimiter':','}
thread_groups:-thread_group_name:线程组1comments:线程组1描述enabled:'true'num_threads:50loops:-1ramp_time:0scheduler:'true'duration:30delay:''http_samples:-request_name:HTTP-GETenabled:'true'csv_files:- {'name':'数据文件4', 'path':'data.csv', 'varnames':'a,b', 'delimiter':','}
request:protocol:httpsdomain:httpbin.orgport:''encoding:''path:/getmethod:GETconnect_timeout:''response_timeout:''params: {'a':1, 'b':2}
headers: {'token':'aaaaaa', 'x-text':'bbbbb'}
follow_redirects:'false'auto_redirects:'false'use_keepalive:'false'validate:-test_field:response_data# response_data响应文本 response_code响应代码response_message响应信息response_headers# 请求头request_headers sample_label URL样本 response_data_as_document文档(文本) 请求数据request_datatest_type:2# 2 包括 1匹配 8 相等 16字符串 否+4 或者+32strings: ['a', 'b']
-request_name:HTTP-POSTenabled:'true'request:protocol:httpsdomain:httpbin.orgport:''encoding:''path:/postmethod:POSTdata: {'c':3, 'd':4}
follow_redirects:'false'auto_redirects:'false'use_keepalive:'false'connect_timeout:''response_timeout:''-request_name:HTTP-JSONenabled:'true'request:protocol:httpsdomain:httpbin.orgport:''encoding:''path:/postmethod:POSTconnect_timeout:''response_timeout:''raw_data:'{"e": 5, "f": 6}'follow_redirects:'false'auto_redirects:'false'use_keepalive:'false'-thread_group_name:线程组2comments:线程组2描述enabled:'false'num_threads:50loops:-1ramp_time:0scheduler:'true'duration:30delay:''csv_files:- {'name':'数据文件3', 'path':'data.csv', 'varnames':'a,b','delimiter':'\t'}
dubbo_samples:-request_name:查询运费接口-dubboenabled:'true'registry:type:zookeepergroup:''address:'zk-mall1.secoolocal.com:5181?backup=zk-mall2.secoolocal.com:5181,zk-mall3.secoolocal.com:5181'dubbo:timeout:100retires:0group:''connections:100load_balance:randomcluster:failfastservice:'com.secoo.business.config.rpc.service.BusinessConfigStoreService'method:queryFreightheaders:Content-Type:'application/json'params:-type:java.util.Listvalue:${freight_wareHouseId_sendAreaId}-type:java.lang.Stringvalue:110100-type:java.util.Listvalue:${freight_wareHouseId_sendAreaId}validate:-test_field:response_data# response_data响应文本 response_code响应代码response_message响应信息response_headerstest_type:16# 2 包括 1匹配 8 相等 16字符串 否+4 或者+32strings: ['"code": 0']
对应的模板文件tpl.xml代码如下:
Copy<?xml version="1.0" encoding="UTF-8"?><jmeterTestPlanversion="1.2"properties="5.0"jmeter="5.3"><hashTree><TestPlanguiclass="TestPlanGui"testclass="TestPlan"testname="{{test_plan_name}}"enabled="true"><stringPropname="TestPlan.comments">{{comments}}</stringProp><boolPropname="TestPlan.functional_mode">false</boolProp><boolPropname="TestPlan.tearDown_on_shutdown">true</boolProp><boolPropname="TestPlan.serialize_threadgroups">false</boolProp><elementPropname="TestPlan.user_defined_variables"elementType="Arguments"guiclass="ArgumentsPanel"testclass="Arguments"testname="用户定义的变量"enabled="true"><collectionPropname="Arguments.arguments"/></elementProp><stringPropname="TestPlan.user_define_classpath"></stringProp></TestPlan><hashTree>{% if hosts %}
<DNSCacheManagerguiclass="DNSCachePanel"testclass="DNSCacheManager"testname="DNS缓存管理器"enabled="true"><collectionPropname="DNSCacheManager.servers"/><collectionPropname="DNSCacheManager.hosts">{% for host in hosts %}
<elementPropname="las.secoo.com"elementType="StaticHost"><stringPropname="StaticHost.Name">{{host.name}}</stringProp><stringPropname="StaticHost.Address">{{host.address}}</stringProp></elementProp>{% endfor %}
</collectionProp><boolPropname="DNSCacheManager.clearEachIteration">false</boolProp><boolPropname="DNSCacheManager.isCustomResolver">true</boolProp></DNSCacheManager><hashTree/>{% endif %} {% if cookies %}
<CookieManagerguiclass="CookiePanel"testclass="CookieManager"testname="HTTP Cookie管理器"enabled="true"><collectionPropname="CookieManager.cookies"/><boolPropname="CookieManager.clearEachIteration">{{cookies.clear_each_iteration}}</boolProp></CookieManager><hashTree/>{% endif %} {% if csv_files %}{% for csv_file in csv_files %}
<CSVDataSetguiclass="TestBeanGUI"testclass="CSVDataSet"testname="{{csv_file.name}}"enabled="true"><stringPropname="filename">dat/{{csv_file.path}}</stringProp><stringPropname="fileEncoding">UTF-8</stringProp><stringPropname="variableNames">{{csv_file.varnames}}</stringProp><boolPropname="ignoreFirstLine">true</boolProp><stringPropname="delimiter">{{csv_file.delimiter}}</stringProp><boolPropname="quotedData">false</boolProp><boolPropname="recycle">true</boolProp><boolPropname="stopThread">false</boolProp><stringPropname="shareMode">shareMode.group</stringProp></CSVDataSet><hashTree/>{% endfor %}{% endif %}
<ResultCollectorguiclass="StatVisualizer"testclass="ResultCollector"testname="聚合报告"enabled="true"><boolPropname="ResultCollector.error_logging">false</boolProp><objProp><name>saveConfig</name><valueclass="SampleSaveConfiguration"><time>true</time><latency>true</latency><timestamp>true</timestamp><success>true</success><label>true</label><code>true</code><message>true</message><threadName>true</threadName><dataType>true</dataType><encoding>false</encoding><assertions>true</assertions><subresults>true</subresults><responseData>false</responseData><samplerData>false</samplerData><xml>false</xml><fieldNames>true</fieldNames><responseHeaders>false</responseHeaders><requestHeaders>false</requestHeaders><responseDataOnError>true</responseDataOnError><saveAssertionResultsFailureMessage>true</saveAssertionResultsFailureMessage><assertionsResultsToSave>0</assertionsResultsToSave><bytes>true</bytes><sentBytes>true</sentBytes><threadCounts>true</threadCounts><idleTime>true</idleTime></value></objProp><stringPropname="filename"></stringProp></ResultCollector><hashTree/>{% for thread_group in thread_groups %}
<ThreadGroupguiclass="ThreadGroupGui"testclass="ThreadGroup"testname="{{thread_group.thread_group_name}}"enabled="{{thread_group.enabled}}"><stringPropname="TestPlan.comments">{{thread_group.comments}}</stringProp><stringPropname="ThreadGroup.on_sample_error">continue</stringProp><elementPropname="ThreadGroup.main_controller"elementType="LoopController"guiclass="LoopControlPanel"testclass="LoopController"testname="循环控制器"enabled="true"><boolPropname="LoopController.continue_forever">false</boolProp><intPropname="LoopController.loops">{{thread_group.loops}}</intProp></elementProp><stringPropname="ThreadGroup.num_threads">{{thread_group.num_threads}}</stringProp><stringPropname="ThreadGroup.ramp_time">{{thread_group.ramp_time}}</stringProp><boolPropname="ThreadGroup.scheduler">{{thread_group.scheduler}}</boolProp><stringPropname="ThreadGroup.duration">{{thread_group.duration}}</stringProp><stringPropname="ThreadGroup.delay">{{thread_group.delay}}</stringProp><boolPropname="ThreadGroup.same_user_on_next_iteration">false</boolProp></ThreadGroup><hashTree>{% if thread_group.csv_files %}{% for csv_file in thread_group.csv_files %}
<CSVDataSetguiclass="TestBeanGUI"testclass="CSVDataSet"testname="{{csv_file.name}}"enabled="true"><stringPropname="filename">dat/{{csv_file.path}}</stringProp><stringPropname="fileEncoding">UTF-8</stringProp><stringPropname="variableNames">{{csv_file.varnames}}</stringProp><boolPropname="ignoreFirstLine">true</boolProp><stringPropname="delimiter">{{csv_file.delimiter}}</stringProp><boolPropname="quotedData">false</boolProp><boolPropname="recycle">true</boolProp><boolPropname="stopThread">false</boolProp><stringPropname="shareMode">shareMode.group</stringProp></CSVDataSet><hashTree/>{% endfor %}{% endif %} {% if thread_group.http_samples %}{% for http_sample in thread_group.http_samples %}
<HTTPSamplerProxyguiclass="HttpTestSampleGui"testclass="HTTPSamplerProxy"testname="{{http_sample.request_name}}"enabled="{{http_sample.enabled}}"><elementPropname="HTTPsampler.Arguments"elementType="Arguments"guiclass="HTTPArgumentsPanel"testclass="Arguments"testname="用户定义的变量"enabled="true"><collectionPropname="Arguments.arguments">{% if http_sample.request.params %}{% for name, value in http_sample.request.params.items() %}
<elementPropname="{{name}}"elementType="HTTPArgument"><boolPropname="HTTPArgument.always_encode">false</boolProp><stringPropname="Argument.value">{{value}}</stringProp><stringPropname="Argument.metadata">=</stringProp><boolPropname="HTTPArgument.use_equals">true</boolProp><stringPropname="Argument.name">{{name}}</stringProp></elementProp>{% endfor %}{% endif %} {% if http_sample.request.data %}{% for name, value in http_sample.request.data.items() %}
<elementPropname="{{name}}"elementType="HTTPArgument"><boolPropname="HTTPArgument.always_encode">false</boolProp><stringPropname="Argument.value">{{value}}</stringProp><stringPropname="Argument.metadata">=</stringProp><boolPropname="HTTPArgument.use_equals">true</boolProp><stringPropname="Argument.name">{{name}}</stringProp></elementProp>{% endfor %}{% endif %} {% if http_sample.request.raw_data %}
<elementPropname=""elementType="HTTPArgument"><boolPropname="HTTPArgument.always_encode">false</boolProp><stringPropname="Argument.value">{{http_sample.request.raw_data}}</stringProp><stringPropname="Argument.metadata">=</stringProp></elementProp>{% endif %}
</collectionProp></elementProp><stringPropname="HTTPSampler.domain">{{http_sample.request.domain}}</stringProp><stringPropname="HTTPSampler.port">{{http_sample.request.port}}</stringProp><stringPropname="HTTPSampler.protocol">{{http_sample.request.protocol}}</stringProp><stringPropname="HTTPSampler.contentEncoding">{{http_sample.request.encoding}}</stringProp><stringPropname="HTTPSampler.path">{{http_sample.request.path}}</stringProp><stringPropname="HTTPSampler.method">{{http_sample.request.method}}</stringProp><boolPropname="HTTPSampler.follow_redirects">{{http_sample.request.follow_redirects}}</boolProp><boolPropname="HTTPSampler.auto_redirects">{{http_sample.request.auto_redirects}}</boolProp><boolPropname="HTTPSampler.use_keepalive">{{http_sample.request.use_keepalive}}</boolProp><boolPropname="HTTPSampler.DO_MULTIPART_POST">false</boolProp><stringPropname="HTTPSampler.embedded_url_re"></stringProp><stringPropname="HTTPSampler.connect_timeout">{{http_sample.request.connect_timeout}}</stringProp><stringPropname="HTTPSampler.response_timeout">{{http_sample.request.response_timeout}}</stringProp></HTTPSamplerProxy><hashTree>{% if http_sample.request.headers %}
<HeaderManagerguiclass="HeaderPanel"testclass="HeaderManager"testname="HTTP信息头管理器"enabled="true"><collectionPropname="HeaderManager.headers">{% for name, value in http_sample.request.headers.items() %}
<elementPropname=""elementType="Header"><stringPropname="Header.name">{{name}}</stringProp><stringPropname="Header.value">{{value}}</stringProp></elementProp>{% endfor %}
</collectionProp></HeaderManager><hashTree/>{% endif %} {% if http_sample.csv_files %}{% for csv_file in http_sample.csv_files %}
<CSVDataSetguiclass="TestBeanGUI"testclass="CSVDataSet"testname="CSV 数据文件设置"enabled="true"><stringPropname="delimiter">{{csv_file.delimiter}}</stringProp><stringPropname="fileEncoding">UTF_8</stringProp><stringPropname="filename">dat/{{csv_file.path}}</stringProp><boolPropname="ignoreFirstLine">true</boolProp><boolPropname="quotedData">false</boolProp><boolPropname="recycle">true</boolProp><stringPropname="shareMode">shareMode.group</stringProp><boolPropname="stopThread">false</boolProp><stringPropname="variableNames">{{csv_file.varnames}}</stringProp></CSVDataSet><hashTree/>{% endfor %}{% endif %} {% if http_sample.validate %}{% for assertion in http_sample.validate %}
<ResponseAssertionguiclass="AssertionGui"testclass="ResponseAssertion"testname="响应断言"enabled="true"><collectionPropname="Asserion.test_strings">{% if assertion.strings %}{% for string in assertion.strings %}
<stringPropname="97">{{string}}</stringProp>{% endfor %}{% endif %}
</collectionProp><stringPropname="Assertion.custom_message"></stringProp><stringPropname="Assertion.test_field">Assertion.{{assertion.test_field}}</stringProp><boolPropname="Assertion.assume_success">false</boolProp><intPropname="Assertion.test_type">{{assertion.test_type}}</intProp></ResponseAssertion><hashTree/>{% endfor %}{% endif %}
<ResultCollectorguiclass="ViewResultsFullVisualizer"testclass="ResultCollector"testname="察看结果树"enabled="true"><boolPropname="ResultCollector.error_logging">false</boolProp><objProp><name>saveConfig</name><valueclass="SampleSaveConfiguration"><time>true</time><latency>true</latency><timestamp>true</timestamp><success>true</success><label>true</label><code>true</code><message>true</message><threadName>true</threadName><dataType>true</dataType><encoding>false</encoding><assertions>true</assertions><subresults>true</subresults><responseData>false</responseData><samplerData>false</samplerData><xml>false</xml><fieldNames>true</fieldNames><responseHeaders>false</responseHeaders><requestHeaders>false</requestHeaders><responseDataOnError>false</responseDataOnError><saveAssertionResultsFailureMessage>true</saveAssertionResultsFailureMessage><assertionsResultsToSave>0</assertionsResultsToSave><bytes>true</bytes><sentBytes>true</sentBytes><url>true</url><threadCounts>true</threadCounts><idleTime>true</idleTime><connectTime>true</connectTime></value></objProp><stringPropname="filename"></stringProp></ResultCollector><hashTree/></hashTree>{% endfor %}{% endif %} {% if thread_group.dubbo_samples %} {% for dubbo_sample in thread_group.dubbo_samples %}
<io.github.ningyu.jmeter.plugin.dubbo.sample.DubboSampleguiclass="io.github.ningyu.jmeter.plugin.dubbo.gui.DubboSampleGui"testclass="io.github.ningyu.jmeter.plugin.dubbo.sample.DubboSample"testname="{{dubbo_sample.request_name}}"enabled="{{dubbo_sample.enabled}}"><stringPropname="FIELD_DUBBO_REGISTRY_PROTOCOL">{{dubbo_sample.registry.type}}</stringProp><stringPropname="FIELD_DUBBO_REGISTRY_GROUP">{{dubbo_sample.registry.group}}</stringProp><stringPropname="FIELD_DUBBO_RPC_PROTOCOL">dubbo://</stringProp><stringPropname="FIELD_DUBBO_ADDRESS">{{dubbo_sample.registry.address}}</stringProp><stringPropname="FIELD_DUBBO_TIMEOUT">{{dubbo_sample.dubbo.timeout}}</stringProp><stringPropname="FIELD_DUBBO_VERSION"></stringProp><stringPropname="FIELD_DUBBO_RETRIES">{{dubbo_sample.dubbo.retries}}</stringProp><stringPropname="FIELD_DUBBO_GROUP">{{dubbo_sample.dubbo.group}}</stringProp><stringPropname="FIELD_DUBBO_CONNECTIONS">{{dubbo_sample.dubbo.connections}}</stringProp><stringPropname="FIELD_DUBBO_LOADBALANCE">{{dubbo_sample.dubbo.load_balance}}</stringProp><stringPropname="FIELD_DUBBO_ASYNC">sync</stringProp><stringPropname="FIELD_DUBBO_CLUSTER">{{dubbo_sample.dubbo.cluster}}</stringProp><stringPropname="FIELD_DUBBO_INTERFACE">{{dubbo_sample.dubbo.service}}</stringProp><stringPropname="FIELD_DUBBO_METHOD">{{dubbo_sample.dubbo.method}}</stringProp><intPropname="FIELD_DUBBO_METHOD_ARGS_SIZE">1</intProp>{% for param in dubbo_sample.dubbo.params %}
<stringPropname="FIELD_DUBBO_METHOD_ARGS_PARAM_TYPE1">{{param.type}}</stringProp><stringPropname="FIELD_DUBBO_METHOD_ARGS_PARAM_VALUE1">{{param.value}}</stringProp>{% endfor %}
<intPropname="FIELD_DUBBO_ATTACHMENT_ARGS_SIZE">0</intProp><stringPropname="FIELD_DUBBO_CONFIG_CENTER_PROTOCOL"></stringProp><stringPropname="FIELD_DUBBO_CONFIG_CENTER_GROUP"></stringProp><stringPropname="FIELD_DUBBO_CONFIG_CENTER_NAMESPACE"></stringProp><stringPropname="FIELD_DUBBO_CONFIG_CENTER_USER_NAME"></stringProp><stringPropname="FIELD_DUBBO_CONFIG_CENTER_PASSWORD"></stringProp><stringPropname="FIELD_DUBBO_CONFIG_CENTER_ADDRESS"></stringProp><stringPropname="FIELD_DUBBO_CONFIG_CENTER_TIMEOUT"></stringProp><stringPropname="FIELD_DUBBO_REGISTRY_USER_NAME"></stringProp><stringPropname="FIELD_DUBBO_REGISTRY_PASSWORD"></stringProp><stringPropname="FIELD_DUBBO_REGISTRY_TIMEOUT"></stringProp></io.github.ningyu.jmeter.plugin.dubbo.sample.DubboSample><hashTree>{% if dubbo_sample.dubbo.headers %}
<HeaderManagerguiclass="HeaderPanel"testclass="HeaderManager"testname="HTTP信息头管理器"enabled="true"><collectionPropname="HeaderManager.headers">{% for name, value in dubbo_sample.dubbo.headers.items() %}
<elementPropname=""elementType="Header"><stringPropname="Header.name">{{name}}</stringProp><stringPropname="Header.value">{{value}}</stringProp></elementProp>{% endfor %}
</collectionProp></HeaderManager><hashTree/>{% endif %} {% if dubbo_sample.validate %} {% for assertion in dubbo_sample.validate %}
<ResponseAssertionguiclass="AssertionGui"testclass="ResponseAssertion"testname="响应断言"enabled="true"><collectionPropname="Asserion.test_strings">{% if assertion.strings %}{% for string in assertion.strings %}
<stringPropname="97">{{string}}</stringProp>{% endfor %}{% endif %}
</collectionProp><stringPropname="Assertion.custom_message"></stringProp><stringPropname="Assertion.test_field">Assertion.{{assertion.test_field}}</stringProp><boolPropname="Assertion.assume_success">false</boolProp><intPropname="Assertion.test_type">{{assertion.test_type}}</intProp></ResponseAssertion>{% endfor %} {% endif %}
<hashTree/>{% endfor %}{% endif %} {% endfor %}
</hashTree></hashTree></hashTree></jmeterTestPlan>
组装出类似data.yaml格式的数据,并使用jinja2渲染模板即可得到完整的jmx文件
pip install pyyaml jinja2
Copyimport yaml
import jinja2
# 组装或读取数据withopen('data.yaml', encoding='utf-8') as f:
data = yaml.safe_load(f)
# 读取模板withopen('tpl.xml', encoding='utf-8') as f:
tpl = f.read()
# 渲染模板生成jmx
jmx = jinja2.Template(tpl).render(data)
withopen(jmx_file, 'w', encoding='utf-8') as f:
f.write(jmx)
后计
在实际项目中,还涉及数据文件的拷贝,节点环境的部署,脚本的分发,报告的下载等等,可以使用paramiko或者fabric或ansible完成,压测节点数据分发的服务管理。
感谢每一个认真阅读我文章的人,礼尚往来总是要有的,虽然不是什么很值钱的东西,如果你用得到的话可以直接拿走:
这些资料,对于【软件测试】的朋友来说应该是最全面最完整的备战仓库,这个仓库也陪伴上万个测试工程师们走过最艰难的路程,希望也能帮助到你!有需要的小伙伴可以点击下方小卡片领取