vllm-advance

vllm/vllm/v1/core/sched/scheduler.py · ef9339e · L1888–1907 · 固定 commit 上游

这是固定版本源码的只读摘录；可在线阅读，也可通过上游链接查看原文件。

1888        # Advance the number of computed tokens for the request AFTER
1889        # the request is scheduled.
1890        # 1. The scheduler_output of the current step has to include the
1891        #    original number of scheduled tokens to determine input IDs.
1892        # 2. Advance the number of computed tokens here allowing us to
1893        #    schedule the prefill request again immediately in the next
1894        #    scheduling step.
1895        # 3. If some tokens (e.g. spec tokens) are rejected later, the number of
1896        #    computed tokens will be adjusted in update_from_output.
1897        num_scheduled_tokens = scheduler_output.num_scheduled_tokens
1898        for req_id, num_scheduled_token in num_scheduled_tokens.items():
1899            request = self.requests[req_id]
1900            request.num_computed_tokens += num_scheduled_token
1901            request.num_in_flight_tokens += num_scheduled_token
1902            if self.defer_block_free:
1903                # Record the in-flight step, to fence deferred block freeing.
1904                request.last_sched_seq = self.sched_step_seq
1905            request.is_prefill_chunk = request.num_computed_tokens < (
1906                request.num_tokens + request.num_output_placeholders
1907            )