In existing Retrieval-Augmented Generation(RAG) method, the top few results of the retrieval system are used, and the selection of information from these results is typically managed by a large-scale language model(LLM). However, in certain use cases, only the top result should be used as input for the LLM. For instance, when using complex, semi-structured software design documents as an external knowledge source, the LLM may struggle to comprehend and select from multiple document fragments. This can lead to incorrect answers if multiple results are used as input. In this study, we propose a method to improve the recall@1 performance in RAG that uses a group of software design documents as an external knowledge source. We interpret the search task as a multiple-choice question, with the top few fragments from the existing search system’s results serving as the choices, and solve this problem using an LLM. We evaluated both the existing and the proposed method using a search task on a group of software design documents, measuring performance with the recall@1 metric.

文の翻訳では訳者や文脈などの条件により異なる訳文が期待され，機械翻訳タスクのベンチマークに用いられる対訳コーパス内にも一つの原文に複数の参照訳が対応している例が多く存在する．一方で既存の機械翻訳モデルの学習手法ではこうしたコーパス内の構造は明示的には用いられない．そこで本研究では，コーパス内の原文が複数の参照訳を持っているという情報を，機械翻訳モデルの学習に明示的に用いる手法を提案する．評価実験において，提案手法を用いて学習したモデルは旧来の最尤誤差損失のみを用いたモデルに劣る結果となったが，提案手法が文レベル最適輸送損失を用いて学習するモデルの性能向上には寄与することが示された．

This paper investigates the pretrained language model (PLM) specialised in the Japanese legal domain. We create PLMs using different pretraining strategies and investigate their performance across multiple domains. Our findings are (i) the PLM built with general domain data can be improved by further pretraining with domain-specific data, (ii) domain-specific PLMs can learn domain-specific and general word meanings simultaneously and can distinguish them, (iii) domain-specific PLMs work better on its target domain; still, the PLMs retain the information learnt in the original PLM even after being further pretrained with domainspecific data, (iv) the PLMs sequentially pretrained with corpora of different domains show high performance for the later learnt domains.

本論文では日本語の法律分野に特化したBERTモデルを提案する．民事事件判決書コーパスを用い，BERTを一から事前学習するモデルと，既存の汎用日本語BERTに追加事前学習するモデルを作成した．実験より，民事事件判決書を用いたMasked Language Model，Next Sentence Prediction タスクについては既存の汎用日本語BERTに追加事前学習する手法が最も良い正解率を示すことがわかった．