Question: What are the differences between TRIALMASTER and traditional methods during inference?

Traditional methods use a depth-first search (DFS) system that generates multiple candidate tactics at each state and tries them one by one, while TRIALMASTER considers the entire proof history (including failed paths and backtracking information) and generates only one optimal tactic at a time. <PIC> TRIALMASTER operates independently without relying on backtracking systems like DFS or BFS, and produces two types of outputs: Lean tactics and backtrack instructions. During inference, it utilizes the complete history of paths, including previous tactics, states, and failed attempts, though it only uses the first tactic from its output while using Lean as a calculator for determining the next state.


Question: Why does TRIALMASTER trained with longer proofs perform worse?

This difference occurs because when proofs contain too much trial-and-error information, excessive failed paths can degrade the quality of training data. <PIC> Models trained with shorter proofs achieve higher success rates, as longer proofs accumulate excessive trial-and-error information that can detrimentally affect the model's performance.


Question: What is the overall architecture of MuFuzz?

MuFuzz's architecture consists of three main components. <PIC> Sequence-Aware Mutation, Mask-Guided Seed Mutation, and Dynamic-Adaptive Energy Adjustment. The Sequence-Aware Mutation module uses data-flow-based feedback to determine meaningful transaction orders and explore deeper states, while the Mask-Guided Seed Mutation module biases transaction inputs generation to hit target branches, and the Dynamic-Adaptive Energy Adjustment module balances fuzzing resource allocation during execution.


Question: How much do the individual components contribute to MuFuzz's performance?

Each component makes important contributions to MuFuzz's performance. <PIC> In particular, generating meaningful transaction sequences plays the most critical role - without the sequence-aware mutation component, coverage decreases by 18% on small contracts and 26% on large contracts. Without mask-guided seed mutation and dynamic energy adjustment, coverage drops by 9% and 10% on small contracts, and 19% and 25% on large contracts respectively. The components also significantly impact bug detection capability - MuFuzz discovers 14%, 6%, and 11% more bugs on small contracts and 27%, 22%, and 24% more bugs on large contracts with all components enabled compared to versions without each component.


Question: What did the paper, "Understanding In-Context Learning in Transformers and LLMs by Learning to Learn Discrete Functions", discover when experimenting with Teaching Sequences?

The paper found that when using Teaching Sequences, Transformers achieved perfect accuracy after receiving the teaching sequence in all 5 test tasks. <PIC> The accuracy remained at or very close to 100% after receiving the teaching sequence, and notably, even for traditionally challenging tasks like DNFs and sparse parities where Transformers typically struggled in the standard setting, they were able to achieve successful learning outcomes using Teaching Sequences.


Question: In the Direct Evaluation experiment, how did GPT-4's performance differ from other LLMs on Conjunction and Majority tasks?

GPT-4 significantly outperformed other models.  <PIC> While GPT-3.5-Turbo and LLaMA-2 performed similarly to each other and could only match or exceed the nearest neighbor baseline when dimensions were up to 7, GPT-4 demonstrated superior performance by outperforming all other models and even slightly outperforming the nearest neighbor baseline in the challenging 15-dimensional case, where the task remains complex with 2^128 possible Boolean functions.


Question: Who are the main participants in the BCFL system and how do they interact?

the BCFL system mainly includes two types of participants: Model Owner and Clients. Specifically, the Model Owner publishes FL tasks, while Clients are responsible for training local models and broadcasting model updates to the blockchain network. The verified model updates are recorded on the blockchain. <PIC> This approach ensures that the Model Owner can only access model updates rather than raw data, thereby protecting participants' privacy.


Question: When do clients and model owners achieve maximum utility in the BCFL model?

Both parties achieve maximum utility when they both choose optimal strategies. <PIC> This was demonstrated in experiments with 50 clients (each with data size μi = 10) comparing four strategy combinations - both sides optimal, one side random with other optimal, and both random - where the results showed that clients and the Model Owner obtained higher utilities than all other strategy combinations when they both employed optimal strategies.


Question: What are the application domains of LLM4BS?

LLM4BS has five main application domains: Code Auditor, Abnormal Transaction Analysis, Fuzzer, Developer, and Community Participants. <PIC>


Question: What is the workflow of LLM4FUZZ? How does it utilize LLM to guide fuzzing?

LLM4FUZZ represents an innovation in blockchain security that combines the capabilities of large language models with fuzzing methods to proactively discover vulnerabilities that might compromise smart contract integrity. <PIC> By deploying LLMs to intelligently guide the fuzzing process, this technology can more specifically explore areas in smart contracts that are most likely to contain security flaws, which not only simplifies the anomaly detection process but also improves its accuracy and depth.


Question: How does Astute RAG improve RAG system performance across subsets partitioned by retrieval precision, especially in scenarios with conflicting knowledge and in worst-case conditions on RGB, compared to other RAG baselines?

Astute RAG enhances Retrieval-Augmented Generation (RAG) performance across subsets differentiated by retrieval precision, especially under challenging conditions such as conflicting knowledge and worst-case scenarios on RGB. First, in terms of performance across retrieval precision, Astute RAG consistently outperforms traditional RAG baselines across various levels of retrieval precision, as demonstrated in the experimental analysis. Unlike other methods, it maintains high performance in both high and low retrieval precision settings, improving trustworthiness without sacrificing gains at higher precision. Notably, when retrieval precision is extremely low (close to zero), Astute RAG surpasses other RAG variants, which otherwise perform worse than the 'No RAG' baseline. This robustness aligns with the worst-case results on RGB, underscoring Astute RAG's effectiveness in handling imperfect retrieval. <PIC> Second, regarding effectiveness in knowledge conflict resolution, in scenarios with conflicting knowledge between internal (LLM) and external (retrieved) sources, Astute RAG selects the correct answer in approximately 80% of cases, making it the most effective solution for these situations. Additionally, it improves performance even on subsets where neither internal nor external knowledge alone is correct, indicating its capability to synthesize partially correct information from both sources, thus resolving conflicts and yielding accurate answers. <PIC>. Lastly, in the context of worst-case performance on RGB, where all retrieved documents are negative, Astute RAG demonstrates superior robustness to noise compared to other RAG baselines. While other methods fall short of the No RAG baseline in these cases, Astute RAG achieves performance close to No RAG, emphasizing its effectiveness in mitigating the detrimental effects of imperfect retrieval. <PIC> Overall, Astute RAG proves robust and reliable across varied and challenging conditions, providing a more resilient approach to retrieval-augmented generation compared to standard RAG methods.


Question: How does Astute RAG utilize its intermediate outputs and internal knowledge to detect and correct erroneous information in generated responses?

Astute RAG leverages its intermediate outputs and internal knowledge to detect and correct erroneous information in generated responses by iteratively comparing generated content with external retrieved information. In Figure, we illustrate two representative examples of this mechanism in action. <PIC> In the first example, the language model (LLM) operating without retrieval generates an incorrect answer, while Astute RAG with retrieval provides the correct response. Astute RAG accomplishes this by identifying inconsistencies between its generated passage and an external passage, effectively mitigating confirmation bias by cross-referencing and isolating the erroneous information . In the second example, the LLM alone correctly answers the query, while RAG introduces an error due to noisy retrieval results. Here, Astute RAG’s internal knowledge aids in discerning the correct answer within the noisy retrieved data by systematically validating retrieved information against its pre-existing knowledge. This capability enables Astute RAG to resolve discrepancies by assessing the credibility of both internal and external information, thereby refining the accuracy of its final output. By continuously contrasting internal and external sources, Astute RAG can pinpoint conflicts and filter out unreliable content, ensuring that only the most accurate information is presented. This structured verification process enhances the robustness of RAG systems, effectively reducing misinformation in generated responses.