Modifying files of BigCode Evaluation Harness

Đã Đóng Đã đăng vào 3 tháng trước Thanh toán khi bàn giao
Đã Đóng Thanh toán khi bàn giao

Currently, while generate and evaluate on a task using bigcode evaluation harness, it generates a .json file something attached below. The task is to try appending a single hard-coded prompt to prompts in TokenizedDataset in lm_eval/

First, understand what the exact structure of the resulting prompt will be like.

E.g. “Prompt: print hello world \n\n Answer: print(‘hello world’) \n\n Prompt: read a file \n\n Answer: “; vs “Intent: print hello world \n\n Code: print(‘hello world’) \n\n Intent: read a file \n\n Code: “.

a. “# prints hello world”; what separator to use between the prompt and the answer

b. check the docs on huggingface for models - to see if they have examples of ICL (in-context learning)

c. check the papers of the models (codegen) - for any examples how to use ICL with the model

Python Ngôn ngữ tự nhiên GPT-3 Deep Learning NLP

ID dự án: #36717006

Về dự án

2 đề xuất Dự án từ xa 2 tháng trước đang mở

2 freelancer chào giá trung bình₹1300 cho công việc này


Objective Highly energetic, motivated and enthusiastic teenager able to quickly learn and handle varied responsibilities is seeking employment opportunities to further enhance learning abilities and to gain experience Thêm

₹1300 INR trong 7 ngày
(0 Nhận xét)