中文版 → · You are viewing the English version

Platform Model Benchmark Report

Subject: ByteHorizon AI Cloud platform endpoint (OpenAI-compatible) Test date: 2026-07-13 Version: v1.1 (three rounds: WiFi daytime · hotspot control · WiFi evening peak)

1. Executive summary

Check	Result	Notes
Capability test (16-question fixed-answer bank)	✅ Pass	All three models: 16/16 perfect (qwen3.5-flash needs an ample token budget, see §4.3)
Time-to-first-token (TTFT)	✅ Pass	3-model avg 0.78–1.82s, all under the 2s excellent line
Throughput	✅ Pass	42.6–51.1 tokens/s, normal for flash-class models
5-way concurrency	✅ Pass	3 models × 5 requests all returned 200, no rate-limit errors
Network link (three-round control)	✅ Pass	throughput differs <3% between WiFi and phone hotspot, proving the bottleneck is server-side and the data is trustworthy (§5.4)
Evening-peak stability	✅ Pass	the 8pm peak round is only 3–7% below daytime throughput, far above the “≥60% of daytime” bar
Model consistency check	⚠️ Two notes	① kimi-k2.6 is a w4a8 quantized deployment (not full precision); ② qwen3.5-flash uses forced deep-reasoning mode, consuming about 9.7× the tokens of deepseek

In one line: the three models' answer accuracy is indistinguishable (all perfect); the real difference is in speed and cost — on the same 16 questions, output-token usage was deepseek-v4-flash 2,997、kimi-k2.6 12,062、qwen3.5-flash 29,117. Under per-token billing, their cost per answer is roughly 1 : 4 : 9.7。

2. Test environment

Category	Details	Measured value
Hardware	Test host	Standard test host (only issues API requests; not involved in inference, so local specs do not affect results)
Hardware	Notes	the test only sends requests; inference runs server-side, so local specs do not affect results
Software	OS	macOS 12.7.6 (21H1320)
Software	Test program	Python 3.14.5 + requests 2.34.2 (in-house scripts, reproducible per Appendix B)
Network	Access	Primary: home WiFi (5GHz); control: phone cellular hotspot (§5.4)
	Bandwidth	WiFi down 3.27 Mbps / up 15.19 Mbps (measured via macOS networkQuality)
	Endpoint latency (WiFi)	ping ×10: 0% loss, avg 33.5ms (min 15.1 / max 85.5)
	Endpoint latency (hotspot)	ping ×10: 0% loss, avg 80.9ms, high jitter (max 467.5ms)
	TLS handshake	108–179ms (curl ×3)
	Proxy/VPN	confirmed all off (no clash core process); direct connection
Endpoint under test	Interface	`platform API endpoint` (OpenAI-compatible)
Endpoint under test	Model	deepseek-v4-flash · kimi-k2.6 · qwen3.5-flash (/v1/models returned exactly these 3)
Time	Test window	2026-07-13 19:22–19:45 (Singapore time, weekday evening)

On the network's effect: capability scores are entirely network-independent (right/wrong answers don't depend on speed). In the speed metrics, network latency is only ~33ms while model responses are measured in seconds, so the effect is under 5%. Downlink is only 3.27 Mbps, but an LLM text stream is only a few KB/s, so bandwidth is nowhere near the bottleneck. The hotspot-control and evening-peak rounds are both complete (§5.4); throughput differs <8% across the three rounds, confirming the robustness of this report's data.

3. Methodology

3.1 Capability test (16-question fixed-answer bank)

Bank composition: 5 math (incl. 1 AIME-level), 3 logic, 4 code, 2 Chinese general-knowledge, 2 instruction-following — all with a single reference answer (full text in Appendix A).
Answering rules: temperature 0 (no randomness); the last line must be “Answer: X”; graded automatically (exact match on the answer line; JSON parsed then compared; string questions by substring).
Fairness handling: qwen3.5-flash is a forced-reasoning model; in round 1 its 1,024–2,048 token budget was exhausted by thinking and truncated (3/16). Re-run with the budget raised to 16,384 and streaming, it scored 16/16.This report uses the re-run score; both rounds are disclosed (§4.3).

3.2 Performance test

TTFT: streaming request, timed to the first content chunk, averaged over 3 runs per model.
Throughput: generate a ~600-character passage (max 1,200 tokens); computed as completion_tokens ÷ generation time.
Concurrency: 5 simultaneous requests per model; success rate and timing recorded.

4. Capability results

4.1 Total scores

deepseek-v4-flash

16/16

avg 3.8s/q · 187 tokens/q

kimi-k2.6

16/16

avg 13.3s/q · 754 tokens/q

qwen3.5-flash

16/16

re-run · avg 53.2s/q · 1,820 tokens/q

Across all five categories (math/logic/code/Chinese/instruction) all three models are perfect, so capability can't separate them; the difference shows in the timing and token usage below.

4.2 Per-question matrix

ID	Category	Question (summary)	Reference answer	deepseek-v4-flash	kimi-k2.6	qwen3.5-flash*
math-1	Math	Sum of first 100 terms of an arithmetic sequence (a₁=3, d=7)	34950	✅ 5.4s	✅ 9.0s	✅ 56s
math-2	Math	2¹⁰⁰ mod 7	2	✅ 5.0s	✅ 22.1s	✅ 65s
math-3	Math	Count of 1–1000 divisible by 3 or 5	467	✅ 5.3s	✅ 12.9s	✅ 58s
math-4	Math	x+y=10, x²+y²=58, find x³+y³	370	✅ 4.5s	✅ 12.5s	✅ 58s
math-5	Math · competition	Sum of positive integers n making n²+85n+2017 a perfect square (AIME)	195	✅ 13.4s	✅ 83.0s	✅ 203s
logic-1	Logic	5 machines make 5 parts in 5 min; how long for 100 machines to make 100	5	✅ 3.0s	✅ 13.7s	✅ 54s
logic-2	Logic	A mother has 3 children: Damao, Ermao, and…?	Xiaoming	✅ 2.2s	✅ 4.4s	✅ 27s
logic-3	Logic	Look-and-say sequence 1, 11, 21, 1211, 111221 — next term	312211	✅ 3.7s	✅ 7.7s	✅ 42s
code-1	Code	Output of a Python generator-sum snippet	18	✅ 3.4s	✅ 7.7s	✅ 45s
code-2	Code	Output of JavaScript typeof NaN	number	✅ 1.7s	✅ 7.3s	✅ 34s
code-3	Code	Length of a Python list after aliasing	4	✅ 3.7s	✅ 7.5s	✅ 50s
code-4	Code	Output of Python 0.1+0.2==0.3	False	✅ 3.0s	✅ 9.2s	✅ 52s
cn-1	Chinese	Which classical essay contains this line? (Chinese)	Tengwang Pavilion Preface	✅ 2.5s	✅ 4.9s	✅ 42s
cn-2	Chinese	Singapore's founding Prime Minister	Lee Kuan Yew	✅ 1.6s	✅ 4.7s	✅ 43s
inst-1	Instruction	Output only the given JSON object, no extra text	{"name":"test"}	✅ 0.8s	✅ 3.3s	✅ 13s
inst-2	Instruction	Reverse the string 'hello world', output only the result	dlrow olleh	✅ 0.9s	✅ 2.4s	✅ 12s

* qwen3.5-flash uses the re-run score (ample budget + streaming, 4 questions in parallel, so timing may be slightly overstated). deepseek/kimi are round-1, run sequentially.

4.3 Fairness disclosure: qwen3.5-flash's two-round gap

Round 1: 3/16 → re-run: 16/16. This is not a capability issue: qwen3.5-flash forces deep reasoning at this endpoint, emitting thousands of characters of thinking per question first. The round-1 budget (1,024–2,048 tokens) was consumed by thinking, so the answer was truncated and scored 0 (every failed question's token count sat exactly at the budget cap with empty body; the competition question hit a gateway 524 from a reasoning timeout). Re-run with the budget raised to 16,384 and streaming, it answered them all. Commercial note: you can't trade a small budget for fast answers with this model — too small a budget means “no answer”, not “short answer”; integrators must reserve a large max_tokens.

4.4 Average time per question (measured)

Average response time per question (seconds, lower is better)

Mean over 16 questions; includes model thinking time. The hard competition item alone: deepseek 13.4s / kimi 83s / qwen 203s

5. Performance results

5.1 TTFT (from hitting enter to the first character)

Time-to-first-token (seconds, lower is better) · avg of 3 runs

Commercial guide: <2s excellent, 2–5s acceptable. kimi's first request 3.41s includes cold start; the next two were 0.72/1.32s

5.2 Throughput (output speed on long generation)

Throughput (tokens/s, higher is better) · 600-character generation task

flash-class normal range is 40–100+ tokens/s; below 20 suggests a crowded pool or downgrade

5.3 Concurrency test

Model	Concurrency	Success	HTTP status	Per-request time
deepseek-v4-flash	5	5/5	all 200	2.2s
kimi-k2.6	5	5/5	all 200	1.6s
qwen3.5-flash	5	5/5	all 200	3.1–3.2s

5.4 Three-round network control (WiFi daytime · hotspot · WiFi evening peak)

Round (2026-07-13)	deepseek-v4-flash		kimi-k2.6		qwen3.5-flash		5-way
Round (2026-07-13)	TTFT	Thrpt	TTFT	Thrpt	TTFT	Thrpt	5-way
① WiFi daytime 19:44	0.88s	49.3	1.82s†	51.1	0.78s	42.6	15/15
② Phone hotspot 20:04	2.51s‡	46.4	1.23s	50.7	0.77s	42.7	15/15
③ WiFi evening peak 20:18	1.02s	45.7	1.43s	50.3	0.76s	43.3	15/15

Throughput differs <8% across rounds (<3% between the two networks) — output speed is determined server-side, independent of the local network, so this report's speed data reflects true server performance;
no meaningful slowdown at evening peak (after 8pm Beijing time) — throughput only 3–7% below daytime, concurrency still all-success; the pool holds up at peak;
† includes a 3.41s first-request cold start; ‡ due to cellular jitter (this round's ping peaked at 467ms), not a slower model — which is why speed should be read from the WiFi rounds.

Throughput in tokens/s. Raw records retained for all three rounds and available on request.

6. Cost efficiency (per-token billing view)

Average output tokens per question (fewer is cheaper) · same 16 questions

A reasoning model's thinking is also billed as output tokens — at equal capability, token usage directly sets the cost per answer

Model	Total output tokens (16 q)	Avg/q	Relative cost	Hard-item single-q usage
deepseek-v4-flash	2,997	187	1.0× (baseline)	821 tok / 13.4s
kimi-k2.6	12,062	754	4.0×	4,679 tok / 83s
qwen3.5-flash	29,117	1,820	9.7×	6,774 tok / 203s

7. Backend model-consistency check

Model	model field echoed by the API	Self-identification	Verdict
deepseek-v4-flash	`deepseek-v4-flash`	“Latest DeepSeek model, built by DeepSeek”	echo normal; behavior (terse direct answers, no visible reasoning) matches the DeepSeek flash line
kimi-k2.6	`Kimi-K2.6 (w4a8 quantized deployment)`	“Kimi, built by Moonshot AI”	the API echo indicates a w4a8 quantized deployment (4-bit weights / 8-bit activations, compressed deployment), not full precision. No quality loss across these 16 questions; our platform is tracking the accuracy of this quantized deployment with the compute service provider
qwen3.5-flash	`Qwen3.5-122B`	“Qwen3.5, built by Alibaba's Tongyi Lab”	backend is a 122B-parameter version; forced-reasoning mode (even simple questions think for hundreds of tokens first), not currently disableable

Note: self-identification can be altered by a gateway and is only indicative; this report cross-checks three ways — whether the 16-question score matches the model's public benchmark level, behavioral signature, and API echo. All three reach their publicly-claimed flagship level (cf. Artificial Analysis 2026-07: Kimi K2.6 tops the open-weight board, DeepSeek V4 close behind), with no sign of a smaller model masquerading under a flagship name.

About this report: this is a 2026-07-13 measurement snapshot. The items surfaced above (kimi being a w4a8 quantized deployment, qwen's forced reasoning inflating latency and cost) have been reported straight back to the compute service provider, and the relevant model parameters are being tuned. We continuously test and track the models we run — closing the loop the moment an issue appears rather than trusting a one-off — which is why we publish these measured reports to customers.

8. Conclusions & selection guide

Use case	Recommended	Why
Support, translation, high-volume Q&A, agent tool-calling	deepseek-v4-flash	fastest (3.8s/q), cheapest (1.0×), crisp instruction-following
Content creation, general chat, mixed tasks	kimi-k2.6	perfect capability, highest throughput (51.1 tok/s), moderate cost (4.0×); note it is a quantized deployment
Deep reasoning, math, complex analysis (not time-critical)	qwen3.5-flash	solid reasoning (solved the competition item on its own), but slow (53s/q) and expensive (9.7×); reserve a large token budget

Items our platform is tracking with the compute service provider

benchmark-score delta of kimi-k2.6's w4a8 quantization vs full precision;
whether qwen3.5-flash can expose a parameter to disable/tune reasoning (forced reasoning currently inflates cost and latency);
formal concurrency/QPS limits and over-limit behavior (5-way showed no throttling, but the ceiling wasn't probed);
each model's max context length and function-calling support (needed by agent customers);
gateway timeout policy (long non-streaming reasoning requests hit 524; integrators should stream).

To be added later

dedicated long-context & function-calling tests (already in the v2 benchmark); the same method on the GLM series once available.

Appendix A · The 16 questions in full, with reference answers

ID	Question (verbatim)	Reference answer
math-1	Arithmetic sequence with first term 3 and common difference 7 — find the sum of the first 100 terms.	34950
math-2	What is 2^100 mod 7?	2
math-3	How many integers from 1 to 1000 are divisible by 3 or 5?	467
math-4	Given x+y=10 and x²+y²=58, find x³+y³.	370
math-5	Find the sum of all positive integers n making n²+85n+2017 a perfect square. (AIME 2017)	195
logic-1	If 5 machines make 5 parts in 5 minutes, how many minutes for 100 machines to make 100 parts?	5
logic-2	Xiaoming's mother has three children: the first is Damao, the second Ermao — what is the third called?	Xiaoming
logic-3	Sequence: 1, 11, 21, 1211, 111221 — what comes next?	312211
code-1	What does this Python code output? print(sum(i for i in range(10) if i % 3 == 0))	18
code-2	In JavaScript, what does console.log(typeof NaN) print?	number
code-3	What does this Python code output? a=[1,2,3] b=a b.append(4) print(len(a))	4
code-4	In Python, what does print(0.1+0.2==0.3) output?	False
cn-1	Which classical Chinese essay contains the line 「落霞与孤鹜齐飞，秋水共长天一色」?	Tengwang Pavilion Preface
cn-2	Who was Singapore's founding Prime Minister?	Lee Kuan Yew
inst-1	Output only a JSON object with key "name" = "test". No other text, explanation, or code fences.	{"name":"test"}
inst-2	Reverse the string 'hello world' entirely; output only the reversed result, nothing else.	dlrow olleh

Except for inst-1/inst-2, every question appended a uniform instruction: “Think step by step; the last line must read only: Answer: <your final answer>”. The grader takes the last “Answer:” line for exact matching.

Appendix B · Raw data & reproducibility

All raw responses and test scripts from the three speed rounds (WiFi daytime / hotspot control / WiFi evening peak) and the capability test are retained and available on request for independent third-party verification.
The test program depends only on a standard HTTP library; grading is against a single reference answer, with temperature 0 removing randomness.