ibm-granite/granite-3.1-8b-instruct 🔢📝 → 📝

⭐ Official ▶️ 770.9K runs 📅 Dec 2024 ⚙️ Cog v0.10.0-alpha27+dev 🔗 GitHub ⚖️ License

code-generation question-answering text-generation text-translation

About

Granite-3.1-8B-Instruct is a lightweight and open-source 8B parameter model is designed to excel in instruction following tasks such as summarization, problem-solving, text translation, reasoning, code tasks, function-calling, and more.

Example Output

Prompt:

"Please list an IBM Research laboratory located in the United States."

Output

One of IBM Research's laboratories located in the United States is the IBM Thomas J. Watson Research Center. This facility is situated in Yorktown Heights, New York, approximately 38 miles north of New York City. Established in 1956, it's one of the oldest corporate-backed research centers in the world. The center has played a pivotal role in numerous technological advancements, including the development of the relational database, the Scanning Tunneling Microscope, high-capacity data storage, and the Java programming language, among others.

Performance Metrics

3.01s Prediction Time

50.31s Total Time

All Input Parameters

{
  "top_k": 50,
  "top_p": 0.9,
  "prompt": "Please list an IBM Research laboratory located in the United States.",
  "max_tokens": 512,
  "min_tokens": 0,
  "temperature": 0.6,
  "system_prompt": "You are an expert in tech history and can answer questions eloquently about the milestones of software organizations.",
  "presence_penalty": 0,
  "frequency_penalty": 0
}

Input Parameters

top_k Type: integerDefault: 50: The number of highest probability tokens to consider for generating the output. If > 0, only keep the top k tokens with highest probability (top-k filtering).
top_p Type: numberDefault: 0.9: A probability threshold for generating the output. If < 1.0, only keep the top tokens with cumulative probability >= top_p (nucleus filtering). Nucleus filtering is described in Holtzman et al. (http://arxiv.org/abs/1904.09751).
prompt Type: stringDefault:: Prompt
max_tokens Type: integerDefault: 512: The maximum number of tokens the model should generate as output.
min_tokens Type: integerDefault: 0: The minimum number of tokens the model should generate as output.
temperature Type: numberDefault: 0.6: The value used to modulate the next token probabilities.
system_prompt Type: stringDefault: You are a helpful assistant.: System prompt to send to the model. This is prepended to the prompt and helps guide system behavior. Ignored for non-chat models.
stop_sequences Type: string: A comma-separated list of sequences to stop generation at. For example, '<end>,<stop>' will stop generation at the first instance of 'end' or '<stop>'.
presence_penalty Type: numberDefault: 0: Presence penalty
frequency_penalty Type: numberDefault: 0: Frequency penalty

Output Schema

Output

Type: array • Items Type: string

Example Execution Logs

INFO:     ::1:45020 - "POST /v1/chat/completions HTTP/1.1" 200 OK
Generation took 3.01sFormatted prompt: Please list an IBM Research laboratory located in the United States.

Version Details

Version ID: b66517f8a5c77dc2ceb16664440faeed91f0ba0470db40a3018006ca0204bd38
Version Created: December 19, 2024

Run on Replicate →