organisciak/ocsai-llama2-7b 🔢✓📝 → 📝

▶️ 2.3K runs 📅 Jan 2024 ⚙️ Cog 0.8.6

text-generation

About

Example Output

Prompt:

"AUT Prompt:brick
Response:build bookshelves
Score:"

Output

Performance Metrics

0.13s Prediction Time

0.17s Total Time

All Input Parameters

{
  "prompt": "AUT Prompt:brick\nResponse:build bookshelves \nScore:\n\n",
  "temperature": 0.01
}

Input Parameters

seed Type: integer: Random seed. Leave blank to randomize the seed
debug Type: booleanDefault: false: provide debugging output in logs
top_p Type: numberDefault: 0.95Range: 0 - 1: When decoding text, samples from the top p percentage of most likely tokens; lower to ignore less likely tokens
prompt (required) Type: string: Prompt to send to the model.
temperature Type: numberDefault: 0.7Range: 0.01 - 5: Adjusts randomness of outputs, greater than 1 is random and 0 is deterministic, 0.75 is a good starting value.
return_logits Type: booleanDefault: false: if set, only return logits for the first token. only useful for testing, etc.
max_new_tokens Type: integerDefault: 128Range: 1 - ∞: Maximum number of tokens to generate. A word is generally 2-3 tokens
min_new_tokens Type: integerDefault: -1Range: -1 - ∞: Minimum number of tokens to generate. To disable, set to -1. A word is generally 2-3 tokens.
stop_sequences Type: string: A comma-separated list of sequences to stop generation at. For example, '<end>,<stop>' will stop generation at the first instance of 'end' or '<stop>'.
replicate_weights Type: string: Path to fine-tuned weights produced by a Replicate fine-tune job.
repetition_penalty Type: numberDefault: 1.15Range: 0 - ∞: A parameter that controls how repetitive text can be. Lower means more repetitive, while higher means less repetitive. Set to 1.0 to disable.

Output Schema

Output

Type: array • Items Type: string

Example Execution Logs

Your formatted prompt is:
AUT Prompt:brick
Response:build bookshelves
Score:
correct lora is already loaded
Overall initialize_peft took 0.000
Exllama: False
INFO 01-10 18:31:56 async_llm_engine.py:371] Received request 0: prompt: 'AUT Prompt:brick\nResponse:build bookshelves \nScore:\n\n', sampling params: SamplingParams(n=1, best_of=1, presence_penalty=0.0, frequency_penalty=1.0, temperature=0.01, top_p=0.95, top_k=50, use_beam_search=False, length_penalty=1.0, early_stopping=False, stop=['</s>'], ignore_eos=False, max_tokens=128, logprobs=None, skip_special_tokens=True), prompt token ids: None.
INFO 01-10 18:31:56 async_llm_engine.py:111] Finished request 0.
hostname: model-hs-77dde5d6-e494c37392ea209c-gpu-a40-5fcfffb44d-5wvkg

Version Details

Version ID: b00751d00cca65ff9213aea7d4fc79b9f91d2af25c5f097bd2d9fd29cc952218
Version Created: January 10, 2024

Run on Replicate →