Add metadata and improve model card for industrial code intelligence

README.md

@@ -1,17 +1,33 @@
+---
+license: apache-2.0
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+- code
+- industrial-code
+- verilog
+- cuda
+- triton
+- chip-design
+- cad
+---
+
 # InCoder-32B: Code Foundation Model for Industrial Scenarios
 
 <div align="center">
 
 [![HuggingFace](https://img.shields.io/badge/🤗-Model%20Hub-yellow)](https://huggingface.co/Multilingual-Multimodal-NLP/IndustrialCoder)
 [![GitHub](https://img.shields.io/badge/GitHub-Industrial--Coder-blue)](https://github.com/CSJianYang/Industrial-Coder)
-[![arXiv](https://img.shields.io/badge/arXiv-2603.16790-red)](https://arxiv.org/abs/2603.16790)
+[![arXiv](https://img.shields.io/badge/arXiv-2603.16790-red)](https://huggingface.co/papers/2603.16790)
 [![License](https://img.shields.io/badge/License-Apache%202.0-green)](LICENSE)
 
 </div>
 
 ## Model Summary
 
-**InCoder-32B** (Industrial-Coder-32B) is the first 32B-parameter code foundation model purpose-built for industrial code intelligence. While general-purpose code LLMs excel at mainstream software tasks, they struggle with the unique demands of industrial programming — hardware semantics, specialized language constructs, strict resource constraints, and domain-specific correctness verification. InCoder-32B unifies code intelligence across five industrial domains:
+**InCoder-32B** (Industrial-Coder-32B) is the first 32B-parameter code foundation model purpose-built for industrial code intelligence. While general-purpose code LLMs excel at mainstream software tasks, they often struggle with the unique demands of industrial programming — hardware semantics, specialized language constructs, strict resource constraints, and domain-specific correctness verification. 
+
+Presented in the paper [InCoder-32B: Code Foundation Model for Industrial Scenarios](https://huggingface.co/papers/2603.16790), InCoder-32B unifies code intelligence across five industrial domains:
 
 | Domain | Languages & Frameworks |
 |---|---|
@@ -21,7 +37,7 @@
 | 🔨 **Compiler Optimization** | x86-64 ASM, C/C++, LLVM-IR |
 | 📐 **3D Modeling / CAD** | CadQuery, OpenCascade, Python |
 
-InCoder-32B achieves competitive general-purpose performance while establishing the strongest open-source baselines across all evaluated industrial domains.
+InCoder-32B achieves highly competitive performance on general tasks while establishing the strongest open-source baselines across all evaluated industrial domains.
 
 ---
 
@@ -37,26 +53,17 @@ InCoder-32B achieves competitive general-purpose performance while establishing
 | HumanEval+ | **89.6%** |
 | MBPP+ | **78.3%** |
 | BigCodeBench (Full) | **49.8%** |
-| τ²-bench (Retail) | **85.1%** |
-| τ²-bench (Telecom) | **86.8%** |
 
 ### Industrial Code Benchmarks
 
 | Benchmark | Domain | InCoder-32B | Best Competing Open-Weight |
 |---|---|---|---|
 | VeriScope Score | Chip Design | **80.7** | 83.2 (GLM-5) |
-| VeriRepair Fix | Chip Design | **80.0%** | 90.0% (GLM-5) |
-| RealBench Syn@1 (Module) | Chip Design | **74.8%** | 50.1% (Kimi-K2-Instruct) |
-| ArchXBench (n) | Chip Design | **51.0** | 50.0 (Claude-Sonnet-4.6) |
 | CAD-Coder Compile | 3D Modeling | **82.0%** | 48.0% (Kimi-K2-Thinking) |
-| CAD-Coder IoU | 3D Modeling | **53.5** | 20.0 (Kimi-K2-Thinking) |
-| EmbedCGen Main | Code Optimization | **35.2%** | 90.2% (GLM-5) |
 | KernelBench L1 | GPU Optimization | **22.2%** | 16.2% (GLM-5) |
 | KernelBench L2 | GPU Optimization | **36.0%** | 28.0% (KernelBench L2) |
-| KernelBench L3 | GPU Optimization | **14.0%** | 8.0% (MiniMax-M2.5) |
-| TritonBench G-call | GPU Optimization | **18.5%** | 28.8% (Claude-Sonnet-4.6) |
 
-> InCoder-32B leads all open-weight baselines on CAD-Coder and KernelBench (all three levels), and even surpasses the proprietary Claude-Sonnet-4.6 on CAD-Coder IoU and KernelBench L1/L2/L3.
+> InCoder-32B leads all open-weight baselines on CAD-Coder and KernelBench (all three levels), and even surpasses proprietary models like Claude-Sonnet-4.6 on CAD-Coder IoU and KernelBench L1/L2/L3.
 
 ---
 
@@ -69,98 +76,28 @@ InCoder-32B adopts a standard decoder-only Transformer architecture with the fol
 | Parameters | ~32B |
 | Layers | 64 |
 | Hidden Size | 5,120 |
-| Intermediate Size | 27,648 |
-| Attention Heads | 40 |
-| KV Heads (GQA) | 8 |
-| Head Dimension | 128 |
-| Vocabulary Size | 76,800 |
 | Max Context Length | 131,072 (128K) |
-| Activation | SiLU |
 | Positional Encoding | RoPE (θ = 500,000) |
 | Precision | BFloat16 |
-| Tie Embeddings | No |
 
 ---
 
-## Training Pipeline
+## Training Pipeline: Code-Flow
 
 InCoder-32B is trained through a three-stage **Code-Flow** pipeline:
 
 ### Stage 1 — Pre-training & Annealing
-
-Industrial code corpora (Verilog, CUDA, firmware C, CadQuery scripts) are severely underrepresented in existing datasets like The Stack v2. We construct a dedicated data pipeline using:
-
-- **Three-step domain recall**: rule-based filtering (file extensions, keywords like `endmodule`, `__global__`), FastText classifier, and semantic encoder retrieval
-- **OCR extraction** from technical books and hardware reference manuals
-- **Multi-level deduplication**: exact hash, MinHash LSH, repository-level fork consolidation, cross-source dedup
-- **Domain-specific validation**: AST comparison, re-compilation, synthesis checks
-- **Data refinement**: normalized formatting, cross-file dependency resolution, code-text alignment annotations
-
-Training details:
-- **Hardware**: 4,096 GPUs
-- **Objectives**: Autoregressive LM + Fill-in-the-Middle (FIM)
-- **Learning rate**: 3 × 10⁻⁴ (constant)
-- **Batch size**: 2,048 globally
-- **Total tokens**: 15T
-- **Curriculum**: function-level → file-level → multi-file/project-level
+- **Industrial Recall**: Data pipeline using rule-based filtering, FastText classifiers, and semantic retrieval for Verilog, CUDA, firmware C, and CadQuery.
+- **Refinement**: OCR extraction from technical manuals, multi-level deduplication, and repository-level fork consolidation.
+- **Training**: 15T total tokens using Autoregressive LM + Fill-in-the-Middle (FIM) objectives.
 
 ### Stage 2 — Mid-Training (Context Extension)
-
-Context window is extended progressively from 8K to 128K tokens across two sub-stages, combined with domain-specific data synthesis:
-
-**Stage 2.1 — 8K → 32K:**
-- Targets file-level tasks: completing RTL modules, infilling kernel functions, generating testbenches
-- Data mix: reasoning QA (40%), agent trajectories (20%), commits (15%), industrial artifacts (15%), FIM (10%)
-
-**Stage 2.2 — 32K → 128K:**
-- Unlocks long-context capabilities: extended debugging sessions, cross-module projects
-- Graduated warm-up: long sequences start at 10%, linearly increases to 50%
-- Data mix shifts toward long-context: agent trajectories (30%), FIM (25%), reasoning QA (25%)
-
-**Synthetic Industrial QA Pipeline:**
-1. *Scenario specification* — identified with practicing hardware engineers
-2. *Seed code generation* — realistic RTL patterns, CUDA memory access idioms, interrupt-driven firmware
-3. *QA synthesis with automated verification* — code execution validation, static analysis, logical consistency checks
+Context window extended progressively from 8K to 128K tokens:
+- **8K → 32K**: Targets file-level tasks like completing RTL modules or kernel functions.
+- **32K → 128K**: Unlocks long-context capabilities for extended debugging and cross-module projects.
 
 ### Stage 3 — Post-Training
-
-2.5M supervised fine-tuning samples are constructed directly from real industrial coding tasks with execution-grounded verification across four environments:
-
-| Environment | Toolchain |
-|---|---|
-| Chip Design | Icarus Verilog, Verilator, Yosys |
-| GPU Optimization | NVIDIA A100, nvcc, Triton compiler |
-| 3D Modeling | CadQuery, OpenCascade |
-| Embedded Systems | arm-none-eabi-gcc, Renode simulator (STM32F407) |
-
-SFT samples are organized into three categories:
-- **Direct solution** — requirement-to-implementation
-- **Defect repair** — failure-feedback-fix loop with closed-loop repair trajectories
-- **Performance & structural optimization** — improving correct code for efficiency, readability, or architecture
-
----
-
-## Benchmarks
-
-### Industrial Benchmarks (New)
-
-This release introduces several new industrial benchmarks:
-
-- **VeriScope** — 568 Verilog generation problems across 5 difficulty levels (combinational logic → dual-core out-of-order RISC-V SoC with cache coherence). Graded via RTL simulation.
-- **VeriRepair** — ~22K training / 300 test Verilog bug-fix samples with 4 major error categories and 20 error types.
-- **EmbedCGen** — 500 bare-metal embedded C generation problems for STM32F407 (ARM Cortex-M4), evaluated via cross-compilation + Renode simulation.
-
-### General Benchmarks Evaluated
-
-| Category | Benchmarks |
-|---|---|
-| Code Generation | EvalPlus (HumanEval, MBPP), BigCodeBench, FullStackBench |
-| Code Reasoning | CRUXEval, LiveCodeBench |
-| Code Efficiency | Mercury |
-| Text-to-SQL | Spider, BIRD |
-| Agentic Coding | Terminal-Bench v1/v2, SWE-bench Verified |
-| Tool Use | Mind2Web, BFCL v3, τ²-bench |
-| Industrial | VeriScope, VeriRepair, RealBench, ArchXBench, CAD-Coder, EmbedCGen, SuperCoder, TritonBench, KernelBench |
+2.5M supervised fine-tuning (SFT) samples constructed from real industrial tasks with execution-grounded verification using toolchains like Icarus Verilog, `nvcc`, and Renode (STM32 simulator).
 
 ---
 
@@ -220,42 +157,16 @@ outputs = model.generate(**inputs, max_new_tokens=256)
 print(tokenizer.decode(outputs[0], skip_special_tokens=True))
 ```
 
-### Chat / Instruction Format
-
-```python
-messages = [
-    {
-        "role": "user",
-        "content": "Optimize this CUDA matrix multiplication kernel for an NVIDIA A100 using shared memory tiling with TILE_SIZE=32."
-    }
-]
-
-text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
-inputs = tokenizer(text, return_tensors="pt").to(model.device)
-outputs = model.generate(**inputs, max_new_tokens=2048, temperature=0.1, do_sample=True)
-print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
-```
-
 ---
 
-## Limitations & Known Failure Modes
-
-Based on analysis of 1,882 failure cases across 9 industrial benchmarks:
-
-- **Compilation & syntax errors**: Dominant in Verilog tasks — 71% of RealBench failures involve malformed literals, incorrect port declarations, or bit-width mismatches.
-- **Incomplete API knowledge**: 47% of EmbedCGen failures are linker errors from undefined or incorrectly typed HAL/CMSIS functions; 33% of TritonBench failures are NameErrors from incorrect Triton API usage.
-- **Format compliance**: 46% of VeriScope failures are unparseable structured outputs where the required format is ignored entirely.
-- **Functional correctness under precise semantics**: 79% of VeriRepair failures produce compilable but functionally incorrect code; most CAD-Coder failures stem from systematic Euler angle convention misinterpretation.
-- **Optimization gap**: 33% of KernelBench failures produce functionally correct but insufficiently fast GPU kernels; 83% of SuperCoder failures result in the model copying input assembly without modification.
-
----
+## Limitations & Disclaimers
 
-## Ablation Findings
+Based on failure analysis, the model may struggle with:
+- **API Knowledge**: Linker errors from undefined HAL/CMSIS functions in embedded C.
+- **Functional Semantics**: Producing compilable but functionally incorrect RTL under complex logic scenarios.
+- **Optimization**: Correct but sub-optimal GPU kernel performance.
 
-- **Repository transition data** outperforms static snapshots for planning tasks
-- **Mid-training reasoning trajectories** improve robustness under distribution shift
-- **Thinking paths** unlock emergent capabilities absent in standard instruction tuning
-- **Scaling industrial SFT data** is a reliable performance driver across all 9 industrial benchmarks (83M → 167M → 250M tokens shows consistent improvement)
+Always review and test generated code in a sandboxed environment. Industrial code (RTL, embedded firmware) requires expert review before deployment.
 
 ---
 
@@ -270,18 +181,4 @@ Based on analysis of 1,882 failure cases across 9 industrial benchmarks:
   journal={arXiv preprint arXiv:2603.16790},
   year={2026}
 }
-```
-
----
-
-## Model Card Authors
-
-Jian Yang, Wei Zhang, Jiajun Wu, Junhang Cheng, Shawn Guo, Haowen Wang, Weicheng Gu, Yaxin Du, Joseph Li, Fanglin Xu, Yizhi Li, Lin Jing, Yuanbo Wang, Yuhan Gao, Ruihao Gong, Chuan Hao, Ran Tao, Aishan Liu, Tuney Zheng, Ganqu Cui, Zhoujun Li, Mingjie Tang, Chenghua Lin, Wayne Xin Zhao, Xianglong Liu, Ming Zhou, Bryan Dai, Weifeng Lv
-
-Affiliations: Beihang University, IQuest Research, Shanghai Jiao Tong University, ELLIS, University of Manchester, Shanghai Artificial Intelligence Laboratory, Sichuan University, Gaoling School of Artificial Intelligence (Renmin University of China), Langboat
-
----
-
-## License
-
-This model is released under the [Apache 2.0 License](LICENSE).
+```