Alien Top
  • Communities
  • Create Post
  • heart
    Support Lemmy
  • search
    Search
  • Login
  • Sign Up
rihard7854B to LocalLLaMA@poweruser.forumEnglish · 1 year ago

NVidia H200 achieves nearly 12,000 tokens/sec on Llama2-13B with TensorRT-LLM

github.com

external-link
message-square
24
fedilink
1
external-link

NVidia H200 achieves nearly 12,000 tokens/sec on Llama2-13B with TensorRT-LLM

github.com

rihard7854B to LocalLLaMA@poweruser.forumEnglish · 1 year ago
message-square
24
fedilink
  • ninjasaid13B
    link
    fedilink
    English
    arrow-up
    1    ·
    1 year ago

    H200? Theres a new accelerator?

LocalLLaMA@poweruser.forum

localllama@poweruser.forum

Subscribe from Remote Instance

Create a post
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !localllama@poweruser.forum

Community to discuss about Llama, the family of large language models created by Meta AI.

Visibility: Public
globe

This community can be federated to other instances and be posted/commented in by their users.

  • 1 user / day
  • 1 user / week
  • 1 user / month
  • 1 user / 6 months
  • 3 local subscribers
  • 3 subscribers
  • 1.02K Posts
  • 5.96K Comments
  • Modlog
  • mods:
  • communick@poweruser.forum
  • BE: 0.19.5
  • Modlog
  • Instances
  • Docs
  • Code
  • join-lemmy.org