lucataco/dolphin-2.9-llama3-8b 🔢📝 → 📝

▶️ 5.3K runs 📅 Jul 2024 ⚙️ Cog 0.10.0-alpha11

About

Dolphin-2.9 has a variety of instruction, conversational, and coding skills. It also has initial agentic abilities and supports function calling

Example Output

Prompt:

"Who are you?"

Output

I am Dolphin, a helpful and friendly AI assistant. How can I assist you today?

Performance Metrics

0.56s Prediction Time

53.75s Total Time

All Input Parameters

{
  "top_k": 50,
  "top_p": 0.9,
  "prompt": "Who are you?",
  "max_tokens": 512,
  "min_tokens": 0,
  "temperature": 0.6,
  "system_prompt": "The assistant is named Dolphin. A helpful and friendly AI assistant, Dolphin avoids discussing the system message unless directly asked about it.",
  "presence_penalty": 0,
  "frequency_penalty": 0
}

Input Parameters

top_k Type: integerDefault: 50: The number of highest probability tokens to consider for generating the output. If > 0, only keep the top k tokens with highest probability (top-k filtering).
top_p Type: numberDefault: 0.9: A probability threshold for generating the output. If < 1.0, only keep the top tokens with cumulative probability >= top_p (nucleus filtering). Nucleus filtering is described in Holtzman et al. (http://arxiv.org/abs/1904.09751).
prompt Type: stringDefault:: Prompt
max_tokens Type: integerDefault: 512: The maximum number of tokens the model should generate as output.
min_tokens Type: integerDefault: 0: The minimum number of tokens the model should generate as output.
temperature Type: numberDefault: 0.6: The value used to modulate the next token probabilities.
system_prompt Type: stringDefault: You are a helpful assistant.: System prompt to send to the model. This is prepended to the prompt and helps guide system behavior. Ignored for non-chat models.
stop_sequences Type: string: A comma-separated list of sequences to stop generation at. For example, '<end>,<stop>' will stop generation at the first instance of 'end' or '<stop>'.
presence_penalty Type: numberDefault: 0: Presence penalty
frequency_penalty Type: numberDefault: 0: Frequency penalty

Output Schema

Output

Type: array • Items Type: string

Example Execution Logs

INFO 07-01 15:57:05 async_llm_engine.py:529] Received request 373796f1566c422ea37a6134ddbdb853: prompt: '<|im_start|>system\nThe assistant is named Dolphin. A helpful and friendly AI assistant, Dolphin avoids discussing the system message unless directly asked about it.<|im_end|>\n<|im_start|>user\nWho are you?<|im_end|>\n<|im_start|>assistant\n', sampling_params: SamplingParams(n=1, best_of=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.6, top_p=0.9, top_k=50, min_p=0.0, seed=None, use_beam_search=False, length_penalty=1.0, early_stopping=False, stop=[], stop_token_ids=[128256], include_stop_str_in_output=False, ignore_eos=False, max_tokens=512, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None), prompt_token_ids: None, lora_request: None.
 stdoutGeneration took 1719849348.36sFormatted prompt: <|im_start|>system
The assistant is named Dolphin. A helpful and friendly AI assistant, Dolphin avoids discussing the system message unless directly asked about it.<|im_end|>
<|im_start|>user
Who are you?<|im_end|>
<|im_start|>assistant
INFO 07-01 15:57:06 async_llm_engine.py:120] Finished request 373796f1566c422ea37a6134ddbdb853.
 stdout

Version Details

Version ID: ee173688d3b8d9e05a5b910f10fb9bab1e9348963ab224579bb90d9fce3fb00b
Version Created: July 1, 2024

Run on Replicate →