Sawyer Task0 — Re-eval (max_steps=1200)

評価予算バグ修正後の再測定（12条件・各50 trials）。表は成功率降順。各セクション右上に成功率。動画は成功回を優先、なければ最遠到達を抜粋。生成: 2026-09-16T06:45:19+09:00 · データ作成メモ追記: 2026-10-05

読み方: scripted-only は接近後〜place で崩れやすい。DAgger あり（特に libero FullLoRA r1）が伸びる。旧280結果との比較は Full FT / scripted-only では無効寄り。 clean50（open-loop成功・noise=0×50）は完了: FullLoRA 8% / VisAct 0% / Full FT 0%（比較基準は s10000·effbs4、near=50/50·grasp=0）。詳細は clean50_gallery。

結果サマリ（成功率つき）

条件	状態	成功率	near	grasp	lift	transport	place	旧評価メモ
libero FullLoRA DAgger r1	完了	38.0% (19/50)	34/50	22/50	22/50	20/50	19/50	旧280: 要参照
libero FullLoRA DAgger r2	完了	30.0% (15/50)	50/50	28/50	28/50	20/50	19/50	旧280: ~16%
libero VisAct DAgger r2	完了	26.0% (13/50)	50/50	19/50	19/50	17/50	12/50
base FullLoRA closefail_v2（正の対照）	完了	22.0% (11/50)	49/50	20/50	20/50	17/50	10/50	旧280: 28%
libero VisAct DAgger r1	完了	8.0% (4/50)	40/50	13/50	13/50	8/50	4/50
base VisAct DAgger 1761	完了	8.0% (4/50)	50/50	7/50	7/50	5/50	4/50
libero VisAct scripted (DAggerなし)	完了	4.0% (2/50)	34/50	11/50	9/50	8/50	4/50	旧280: 低め
base VisAct DAgger r2	完了	4.0% (2/50)	36/50	3/50	3/50	3/50	2/50
libero Full FT (subset50)	完了	0.0% (0/50)	5/50	0/50	0/50	0/50	0/50	旧280: 0%（予算不足で無効）
libero FullLoRA scripted (DAggerなし)	完了	0.0% (0/50)	40/50	14/50	13/50	10/50	0/50	旧280: 0%
base FullLoRA scripted (DAggerなし)	完了	0.0% (0/50)	10/50	1/50	1/50	0/50	0/50	旧280: 0%
base VisAct scripted (DAggerなし)	完了	0.0% (0/50)	0/50	0/50	0/50	0/50	0/50	旧280: 0%

共通の評価条件

robot	sawyer
task	libero_object task0 (alphabet soup → basket)
seed	7
num_trials	50
max_steps	1200
env_horizon	1260
home	home_w30
control_freq_hz	20
action_space	delta_osc
gripper_start	open (task0)
note	旧評価は max_steps=280（Frankaスケール）。今回はデモ長〜1044に合わせて1200に修正。

共通の学習条件（既存12条件）

config	`pi05_lora_b_object_sawyer`
batch_size	4（1 GPU）
train_steps	10000（Full FT のみ 5000）
save_interval	5000
optimizer / lr	デフォルト（末尾付近 lr≈5e-5）
control	delta_osc · 20 Hz · home_w30
scripted base data	`libero_sawyer_scripted_obj0_osc_v8`（871 ep、approach_noise≈0.015 あり）
DAgger	条件により none / r1 / r2 / closefail_v2（下表）

データの作り方（何 ep を学習したか）

ポイント: 「学習に使った ep 数」= 下表の train_ep（LeRobot repo の総エピソード数）。 DAgger 条件は scripted ベースに expert takeover デモを合流（多くは expert 成功エピソードを繰り返しマージ）。

A. Scripted ベース（osc_v8）の作り方

Sawyer · Task0 · scripted 軌道をシミュで実行（grasp=ik_set · path_step≈8mm · compact · home_w30）
接近に approach_noise ≈ 1.5cm × 3 copies（+ clean）を混ぜて多様性を確保
ライブ IK デモを OSC（delta_osc）に retrack して学習用 action に揃える
--require-open-loop: 記録 action の再生で lift+place できるデモだけ残す
結果: libero_sawyer_scripted_obj0_osc_v8 = 871 ep

B. DAgger の作り方

student（前段ポリシー）を closed-loop で走らせる
接近失敗・掴み失敗などのトリガで OSC expert が takeover（pose を引き継ぐ）
expert 成功エピソードのみ採用（--require-expert-success）
merge_lerobot_repos.py で scripted（または前 round）と合流。expert 側は extra-repeat（多くは ×10） で増幅
合流データで再学習（steps=10000 · bs=4）→ LoRA merge → 本評価

C. 条件ごとの学習データ量（実測 ep）

条件	train_ep	データの中身	作り方の要約
libero / base · scripted（DAggerなし）	871	`…osc_v8`	上記 A のみ
libero Full FT subset50	50	`…osc_v8_subset50_s7`	osc_v8 の先頭50本だけ
libero FullLoRA DAgger r1	1351	`…dagger_liberoinit_fulllora_r1`	871 + FullLoRA(nodagger) student の takeover を merge
libero FullLoRA DAgger r2	1581	`…fulllora_r2`	r1 合流データ + r1 ポリシーでの追加 takeover
libero VisAct DAgger r1	1181	`…visact_r1`	871 + VisAct(nodagger) student takeover
libero VisAct DAgger r2	1521	`…visact_r2`	r1 合流 + r1 ポリシーでの追加 takeover
base VisAct DAgger 1761（タグ名）	1249	`…baseinit_visact_dagger_1761`	目標タグは 1761 だが、実 repo は 1249 ep
base VisAct DAgger r2	1719	`…baseinit_visact_dagger_r2`	前段 base VisAct dagger + 追加 takeover
base FullLoRA closefail_v2	1761	`…dagger_closefail_v2`	接近〜掴み失敗（close-fail）トリガの takeover を scripted に合流

条件別の学習設定（既存12）

条件	init	method	DAgger	train_ep	data	steps
libero FullLoRA DAgger r1	pi05_libero	Full LoRA	r1	1351	scripted + DAgger r1	10000
libero FullLoRA DAgger r2	pi05_libero	Full LoRA	r2	1581	r1 merged + DAgger r2	10000
libero VisAct DAgger r2	pi05_libero	VisAct LoRA	r2	1521	r1 merged + DAgger r2	10000
base FullLoRA closefail_v2	pi05_base	Full LoRA	closefail_v2	1761	scripted + closefail DAgger	10000
libero VisAct DAgger r1	pi05_libero	VisAct LoRA	r1	1181	scripted + DAgger r1	10000
base VisAct DAgger 1761	pi05_base	VisAct LoRA	r1（実1249）	1249	scripted + DAgger	10000
libero VisAct scripted	pi05_libero	VisAct LoRA	none	871	scripted OSC v8	10000
base VisAct DAgger r2	pi05_base	VisAct LoRA	r2	1719	r1 merged + DAgger r2	10000
libero Full FT (subset50)	pi05_libero	Full FT	none	50	scripted 先頭50	5000
libero FullLoRA scripted	pi05_libero	Full LoRA	none	871	scripted OSC v8	10000
base FullLoRA scripted	pi05_base	Full LoRA	none	871	scripted OSC v8	10000
base VisAct scripted	pi05_base	VisAct LoRA	none	871	scripted OSC v8	10000

次の実験: clean50（結果あり）

open-loop 成功のみ・approach_noise=0 のデモ×50で再学習した結果: FullLoRA 8% / VisAct 0% / Full FT 0%。 3条件とも step=10000 · effbs=4（Full FT は 2GPU×bs2）。詳細・動画は clean50_gallery。

状態	完了（FullLoRA / VisAct / Full FT すべて max1200 評価済み）
FullLoRA	8.0%（4/50）· near50 grasp18 place7
VisAct	0.0%（0/50）· near40 grasp7 place1
Full FT	0.0%（0/50）· near50 grasp0 · s10000 · effbs4（比較基準）
data	`libero_sawyer_scripted_obj0_osc_v8_clean50`（--require-open-loop 成功のみ · noise=0）
init	pi05_libero
eval	seed=7 · 50 trials · max_steps=1200

条件別・動画（成功率つき）

libero FullLoRA DAgger r1

38.0%

成功率 38.0%（19/50 成功） · init=pi05_libero · Full LoRA · dagger=r1 · data=scripted + DAgger r1 · steps=10000

成功率	`38.0%（19/50 成功）`
ckpt	`pi05_lora_b_object_sawyer_merged_libero_init_sawyer_obj0_fulllora_s10000_dagger_liberoinit_fulllora_r1`
repo / norm	`libero_sawyer_scripted_obj0_osc_v8_dagger_liberoinit_fulllora_r1`
eval_log	`/storage3/tmp/yui.muranaka/openpi_eval/eval_libero_fulllora_r1_max1200.log`
video_dir	`/storage3/tmp/yui.muranaka/openpi_eval/videos_libero_fulllora_r1_max1200`

ep17 · success · min_eef=0.005m · lift=0.239m
flags: success, near, grasp, lift, transport, place

ep29 · success · min_eef=0.007m · lift=0.181m
flags: success, near, grasp, lift, transport, place

ep39 · place · min_eef=0.002m · lift=0.316m
flags: near, grasp, lift, transport, place

ep12 · lift · min_eef=0.005m · lift=0.135m
flags: near, grasp, lift

libero FullLoRA DAgger r2

30.0%

成功率 30.0%（15/50 成功） · init=pi05_libero · Full LoRA · dagger=r2 · data=r1 merged + DAgger r2 · steps=10000

成功率	`30.0%（15/50 成功）`
ckpt	`pi05_lora_b_object_sawyer_merged_libero_init_sawyer_obj0_fulllora_s10000_dagger_liberoinit_fulllora_r2`
repo / norm	`libero_sawyer_scripted_obj0_osc_v8_dagger_liberoinit_fulllora_r2`
eval_log	`/storage3/tmp/yui.muranaka/openpi_eval/eval_libero_fulllora_r2_max1200.log`
video_dir	`/storage3/tmp/yui.muranaka/openpi_eval/videos_libero_fulllora_r2_max1200`

ep25 · success · min_eef=0.005m · lift=0.309m
flags: success, near, grasp, lift, transport, place

ep49 · success · min_eef=0.003m · lift=0.287m
flags: success, near, grasp, lift, transport, place

ep19 · place · min_eef=0.004m · lift=0.313m
flags: near, grasp, lift, transport, place

ep24 · place · min_eef=0.004m · lift=0.276m
flags: near, grasp, lift, transport, place

libero VisAct DAgger r2

26.0%

成功率 26.0%（13/50 成功） · init=pi05_libero · VisAct LoRA · dagger=r2 · data=r1 merged + DAgger r2 · steps=10000

成功率	`26.0%（13/50 成功）`
ckpt	`pi05_lora_b_object_sawyer_merged_libero_init_sawyer_obj0_visact_s10000_dagger_liberoinit_visact_r2`
repo / norm	`libero_sawyer_scripted_obj0_osc_v8_dagger_liberoinit_visact_r2`
eval_log	`/storage3/tmp/yui.muranaka/openpi_eval/eval_libero_visact_r2_max1200.log`
video_dir	`/storage3/tmp/yui.muranaka/openpi_eval/videos_libero_visact_r2_max1200`

ep12 · success · min_eef=0.001m · lift=0.150m
flags: success, near, grasp, lift, transport, place

ep43 · success · min_eef=0.002m · lift=0.149m
flags: success, near, grasp, lift, transport, place

ep10 · transport · min_eef=0.002m · lift=0.146m
flags: near, grasp, lift, transport

ep28 · transport · min_eef=0.002m · lift=0.145m
flags: near, grasp, lift, transport

base FullLoRA closefail_v2（正の対照）

22.0%

成功率 22.0%（11/50 成功） · init=pi05_base · Full LoRA · dagger=closefail_v2 · data=scripted + closefail DAgger · steps=10000

成功率	`22.0%（11/50 成功）`
ckpt	`pi05_lora_b_object_sawyer_merged_bs4_10000_scripted_obj0_osc_v8_dagger_closefail_v2`
repo / norm	`libero_sawyer_scripted_obj0_osc_v8_dagger_closefail_v2`
eval_log	`/storage3/tmp/yui.muranaka/openpi_eval/eval_hist_closefail_v2_max1200.log`
video_dir	`/storage3/tmp/yui.muranaka/openpi_eval/videos_hist_closefail_v2_max1200`

ep37 · success · min_eef=0.006m · lift=0.160m
flags: success, near, grasp, lift, transport, place

ep48 · success · min_eef=0.008m · lift=0.160m
flags: success, near, grasp, lift, transport, place

ep2 · transport · min_eef=0.002m · lift=0.159m
flags: near, grasp, lift, transport

ep19 · transport · min_eef=0.003m · lift=0.145m
flags: near, grasp, lift, transport

libero VisAct DAgger r1

8.0%

成功率 8.0%（4/50 成功） · init=pi05_libero · VisAct LoRA · dagger=r1 · data=scripted + DAgger r1 · steps=10000

成功率	`8.0%（4/50 成功）`
ckpt	`pi05_lora_b_object_sawyer_merged_libero_init_sawyer_obj0_visact_s10000_dagger_liberoinit_visact_r1`
repo / norm	`libero_sawyer_scripted_obj0_osc_v8_dagger_liberoinit_visact_r1`
eval_log	`/storage3/tmp/yui.muranaka/openpi_eval/eval_libero_visact_r1_max1200.log`
video_dir	`/storage3/tmp/yui.muranaka/openpi_eval/videos_libero_visact_r1_max1200`

ep34 · success · min_eef=0.007m · lift=0.151m
flags: success, near, grasp, lift, transport, place

ep1 · success · min_eef=0.004m · lift=0.146m
flags: success, near, grasp, lift, transport, place

ep7 · transport · min_eef=0.003m · lift=0.145m
flags: near, grasp, lift, transport

ep35 · transport · min_eef=0.002m · lift=0.144m
flags: near, grasp, lift, transport

base VisAct DAgger 1761

8.0%

成功率 8.0%（4/50 成功） · init=pi05_base · VisAct LoRA · dagger=r1 (~1761) · data=scripted + DAgger toward ~1761ep · steps=10000

成功率	`8.0%（4/50 成功）`
ckpt	`pi05_lora_b_object_sawyer_merged_base_init_sawyer_obj0_visact_s10000_dagger_baseinit_visact_dagger_1761`
repo / norm	`libero_sawyer_scripted_obj0_osc_v8_dagger_baseinit_visact_dagger_1761`
eval_log	`/storage3/tmp/yui.muranaka/openpi_eval/eval_base_visact_r1_max1200.log`
video_dir	`/storage3/tmp/yui.muranaka/openpi_eval/videos_base_visact_r1_max1200`

ep9 · success · min_eef=0.001m · lift=0.173m
flags: success, near, grasp, lift, transport, place

ep11 · success · min_eef=0.005m · lift=0.171m
flags: success, near, grasp, lift, transport, place

ep30 · transport · min_eef=0.002m · lift=0.125m
flags: near, grasp, lift, transport

ep31 · lift · min_eef=0.006m · lift=0.099m
flags: near, grasp, lift

libero VisAct scripted (DAggerなし)

4.0%

成功率 4.0%（2/50 成功） · init=pi05_libero · VisAct LoRA · dagger=none · data=scripted OSC v8 (871 ep) · steps=10000

成功率	`4.0%（2/50 成功）`
ckpt	`pi05_lora_b_object_sawyer_merged_libero_init_sawyer_obj0_visact_s10000_scripted_obj0_osc_v8`
repo / norm	`libero_sawyer_scripted_obj0_osc_v8`
eval_log	`/storage3/tmp/yui.muranaka/openpi_eval/eval_libero_visact_nodagger_max1200.log`
video_dir	`/storage3/tmp/yui.muranaka/openpi_eval/videos_libero_visact_nodagger_max1200`

ep12 · success · min_eef=0.012m · lift=0.147m
flags: success, near, grasp, lift, transport, place

ep27 · success · min_eef=0.013m · lift=0.144m
flags: success, near, grasp, lift, transport, place

ep2 · place · min_eef=0.000m · lift=0.175m
flags: near, grasp, lift, transport, place

ep11 · place · min_eef=0.004m · lift=0.171m
flags: near, grasp, lift, transport, place

base VisAct DAgger r2

4.0%

成功率 4.0%（2/50 成功） · init=pi05_base · VisAct LoRA · dagger=r2 · data=r1 merged + DAgger r2 · steps=10000

成功率	`4.0%（2/50 成功）`
ckpt	`pi05_lora_b_object_sawyer_merged_base_init_sawyer_obj0_visact_s10000_dagger_baseinit_visact_dagger_r2`
repo / norm	`libero_sawyer_scripted_obj0_osc_v8_dagger_baseinit_visact_dagger_r2`
eval_log	`/storage3/tmp/yui.muranaka/openpi_eval/eval_base_visact_r2_max1200.log`
video_dir	`/storage3/tmp/yui.muranaka/openpi_eval/videos_base_visact_r2_max1200`

ep40 · success · min_eef=0.008m · lift=0.144m
flags: success, near, grasp, lift, transport, place

ep29 · success · min_eef=0.006m · lift=0.140m
flags: success, near, grasp, lift, transport, place

ep10 · transport · min_eef=0.007m · lift=0.117m
flags: near, grasp, lift, transport

ep31 · near · min_eef=0.018m · lift=0.013m
flags: near

libero Full FT (subset50)

0.0%

成功率 0.0%（0/50 成功） · init=pi05_libero · Full FT (no LoRA) · dagger=none · data=scripted OSC v8 subset50 (先頭50本) · steps=5000

成功率	`0.0%（0/50 成功）`
ckpt	`pi05_lora_b_object_sawyer/libero_init_sawyer_obj0_fullft_s5000_subset50_s7/5000`
repo / norm	`libero_sawyer_scripted_obj0_osc_v8_subset50_s7`
eval_log	`/storage3/tmp/yui.muranaka/openpi_eval/eval_fullft_libero_subset50_max1200.log`
video_dir	`/storage3/tmp/yui.muranaka/openpi_eval/videos_fullft_libero_subset50_max1200`

ep6 · near · min_eef=0.001m · lift=0.000m
flags: near

ep42 · near · min_eef=0.012m · lift=0.000m
flags: near

ep25 · near · min_eef=0.021m · lift=0.000m
flags: near

ep8 · near · min_eef=0.023m · lift=0.000m
flags: near

libero FullLoRA scripted (DAggerなし)

0.0%

成功率 0.0%（0/50 成功） · init=pi05_libero · Full LoRA · dagger=none · data=scripted OSC v8 (871 ep) · steps=10000

成功率	`0.0%（0/50 成功）`
ckpt	`pi05_lora_b_object_sawyer_merged_libero_init_sawyer_obj0_fulllora_s10000_scripted_obj0_osc_v8`
repo / norm	`libero_sawyer_scripted_obj0_osc_v8`
eval_log	`/storage3/tmp/yui.muranaka/openpi_eval/eval_libero_fulllora_nodagger_max1200.log`
video_dir	`/storage3/tmp/yui.muranaka/openpi_eval/videos_libero_fulllora_nodagger_max1200`

ep38 · transport · min_eef=0.002m · lift=0.146m
flags: near, grasp, lift, transport

ep8 · transport · min_eef=0.001m · lift=0.141m
flags: near, grasp, lift, transport

ep14 · transport · min_eef=0.001m · lift=0.137m
flags: near, grasp, lift, transport

ep45 · transport · min_eef=0.009m · lift=0.133m
flags: near, grasp, lift, transport

base FullLoRA scripted (DAggerなし)

0.0%

成功率 0.0%（0/50 成功） · init=pi05_base · Full LoRA · dagger=none · data=scripted OSC v8 (871 ep) · steps=10000

成功率	`0.0%（0/50 成功）`
ckpt	`pi05_lora_b_object_sawyer_merged_bs4_10000_scripted_obj0_osc_v8`
repo / norm	`libero_sawyer_scripted_obj0_osc_v8`
eval_log	`/storage3/tmp/yui.muranaka/openpi_eval/eval_base_fulllora_scripted_max1200.log`
video_dir	`/storage3/tmp/yui.muranaka/openpi_eval/videos_base_fulllora_scripted_max1200`

ep40 · lift · min_eef=0.001m · lift=0.165m
flags: near, grasp, lift

ep26 · near · min_eef=0.006m · lift=0.000m
flags: near

ep27 · near · min_eef=0.010m · lift=0.000m
flags: near

ep30 · near · min_eef=0.012m · lift=0.000m
flags: near

base VisAct scripted (DAggerなし)

0.0%

成功率 0.0%（0/50 成功） · init=pi05_base · VisAct LoRA · dagger=none · data=scripted OSC v8 (871 ep) · steps=10000

成功率	`0.0%（0/50 成功）`
ckpt	`pi05_lora_b_object_sawyer_merged_base_init_sawyer_obj0_visact_s10000_scripted_obj0_osc_v8`
repo / norm	`libero_sawyer_scripted_obj0_osc_v8`
eval_log	`/storage3/tmp/yui.muranaka/openpi_eval/eval_base_visact_nodagger_max1200.log`
video_dir	`/storage3/tmp/yui.muranaka/openpi_eval/videos_base_visact_nodagger_max1200`

ep7 · miss · min_eef=0.053m · lift=0.000m
flags: none

ep35 · miss · min_eef=0.054m · lift=0.000m
flags: none

ep1 · miss · min_eef=0.061m · lift=0.000m
flags: none

ep6 · miss · min_eef=0.062m · lift=0.000m
flags: none

Rebuild: python scripts/export_reeval_max1200_gallery.py --out docs/reeval_max1200