ElevenlabsClient
A comprehensive Ruby client library for the ElevenLabs API, supporting voice synthesis, dubbing, dialogue generation, sound effects, AI music composition, voice transformation, speech transcription, audio isolation, and advanced audio processing features.
Features
๐๏ธ Text-to-Speech - Convert text to natural-sounding speech
๐ฌ Dubbing - Create dubbed versions of audio/video content
๐ฌ Dialogue Generation - Multi-speaker conversations
๐ Sound Generation - AI-generated sound effects and ambient audio
๐ต Music Generation - AI-powered music composition and streaming
๐จ Voice Design - Create custom voices from text descriptions
๐ญ Voice Management - Create, edit, and manage individual voices
๐ Speech-to-Speech - Transform audio from one voice to another (Voice Changer)
๐ Speech-to-Text - Transcribe audio and video files with advanced features
๐ Audio Isolation - Remove background noise from audio files
๐ฑ Audio Native - Create embeddable audio players for websites
โฑ๏ธ Forced Alignment - Get precise timing information for audio transcripts
๐ค Models - List available models and their capabilities
๐ก Streaming - Real-time audio streaming
โ๏ธ Configurable - Flexible configuration options
๐งช Well-tested - Comprehensive test coverage
Installation
Add this line to your application's Gemfile:
gem 'elevenlabs_client'
And then execute:
$ bundle install
Or install it yourself as:
$ gem install elevenlabs_client
Quick Start
Configuration
Rails Applications (Recommended)
Create config/initializers/elevenlabs_client.rb
:
ElevenlabsClient::Settings.configure do |config|
config.properties = {
elevenlabs_base_uri: ENV["ELEVENLABS_BASE_URL"],
elevenlabs_api_key: ENV["ELEVENLABS_API_KEY"]
}
end
Set your environment variables:
export ELEVENLABS_API_KEY="your_api_key_here"
export ELEVENLABS_BASE_URL="https://api.elevenlabs.io" # Optional, defaults to official API
Direct Configuration
# Module-level configuration
ElevenlabsClient.configure do |config|
config.properties = {
elevenlabs_base_uri: "https://api.elevenlabs.io",
elevenlabs_api_key: "your_api_key_here"
}
end
# Or pass directly to client
client = ElevenlabsClient.new(
api_key: "your_api_key_here",
base_url: "https://api.elevenlabs.io"
)
Basic Usage
# Initialize client (uses configured settings)
client = ElevenlabsClient.new
# Text-to-Speech
audio_data = client.text_to_speech.convert("21m00Tcm4TlvDq8ikWAM", "Hello, world!")
File.open("hello.mp3", "wb") { |f| f.write(audio_data) }
# Dubbing
File.open("video.mp4", "rb") do |file|
result = client.dubs.create(
file_io: file,
filename: "video.mp4",
target_languages: ["es", "fr", "de"]
)
end
# Dialogue Generation
dialogue = [
{ text: "Hello, how are you?", voice_id: "voice_1" },
{ text: "I'm doing great, thanks!", voice_id: "voice_2" }
]
audio_data = client.text_to_dialogue.convert(dialogue)
# Sound Generation
audio_data = client.sound_generation.generate("Ocean waves crashing on rocks")
# Voice Design
design_result = client.text_to_voice.design("Warm, professional female voice")
generated_voice_id = design_result["previews"].first["generated_voice_id"]
voice_result = client.text_to_voice.create(
"Professional Voice",
"Warm, professional female voice",
generated_voice_id
)
# List Available Models
models = client.models.list
fastest_model = models["models"].min_by { |m| m["token_cost_factor"] }
puts "Fastest model: #{fastest_model['name']}"
# Voice Management
voices = client.voices.list
puts "Total voices: #{voices['voices'].length}"
# Create custom voice from audio samples
File.open("sample1.mp3", "rb") do |sample|
voice = client.voices.create("My Voice", [sample], description: "Custom narrator voice")
puts "Created voice: #{voice['voice_id']}"
end
# Music Generation
music_data = client.music.compose(
prompt: "Upbeat electronic dance track with synthesizers",
music_length_ms: 30000
)
File.open("generated_music.mp3", "wb") { |f| f.write(music_data) }
# Speech-to-Speech (Voice Changer)
File.open("input_audio.mp3", "rb") do |audio_file|
converted_audio = client.speech_to_speech.convert(
"target_voice_id",
audio_file,
"input_audio.mp3",
remove_background_noise: true
)
File.open("converted_audio.mp3", "wb") { |f| f.write(converted_audio) }
end
# Speech-to-Text Transcription
File.open("audio.mp3", "rb") do |audio_file|
transcription = client.speech_to_text.create(
"scribe_v1",
file: audio_file,
filename: "audio.mp3",
diarize: true,
timestamps_granularity: "word"
)
puts "Transcribed: #{transcription['text']}"
end
# Audio Isolation (Background Noise Removal)
File.open("noisy_audio.mp3", "rb") do |audio_file|
clean_audio = client.audio_isolation.isolate(audio_file, "noisy_audio.mp3")
File.open("clean_audio.mp3", "wb") { |f| f.write(clean_audio) }
end
# Audio Native (Embeddable Player)
File.open("article.html", "rb") do |html_file|
project = client.audio_native.create(
"My Article",
file: html_file,
filename: "article.html",
voice_id: "voice_id",
auto_convert: true
)
puts "Player HTML: #{project['html_snippet']}"
end
# Forced Alignment
File.open("speech.wav", "rb") do |audio_file|
alignment = client.forced_alignment.create(
audio_file,
"speech.wav",
"Hello world, this is a test transcript"
)
alignment['words'].each do |word|
puts "#{word['text']}: #{word['start']}s - #{word['end']}s"
end
end
# Streaming Text-to-Speech
client.text_to_speech_stream.stream("voice_id", "Streaming text") do |chunk|
# Process audio chunk in real-time
puts "Received #{chunk.bytesize} bytes"
end
API Documentation
Core APIs
- Dubbing API - Create dubbed versions of audio/video content
- Text-to-Speech API - Convert text to natural speech
- Text-to-Speech Streaming API - Real-time audio streaming
- Text-to-Dialogue API - Multi-speaker conversations
- Sound Generation API - AI-generated sound effects
- Music Generation API - AI-powered music composition and streaming
- Text-to-Voice API - Design and create custom voices
- Voice Management API - Manage individual voices (CRUD operations)
- Speech-to-Speech API - Transform audio from one voice to another
- Speech-to-Text API - Transcribe audio and video files
- Audio Isolation API - Remove background noise from audio
- Audio Native API - Create embeddable audio players
- Forced Alignment API - Get precise timing information
- Models API - List available models and capabilities
Available Endpoints
Endpoint | Description | Documentation |
---|---|---|
client.dubs.* |
Audio/video dubbing | DUBBING.md |
client.text_to_speech.* |
Text-to-speech conversion | TEXT_TO_SPEECH.md |
client.text_to_speech_stream.* |
Streaming TTS | TEXT_TO_SPEECH_STREAMING.md |
client.text_to_dialogue.* |
Dialogue generation | TEXT_TO_DIALOGUE.md |
client.sound_generation.* |
Sound effect generation | SOUND_GENERATION.md |
client.music.* |
AI music composition and streaming | MUSIC.md |
client.text_to_voice.* |
Voice design and creation | TEXT_TO_VOICE.md |
client.voices.* |
Voice management (CRUD) | VOICES.md |
client.speech_to_speech.* |
Voice changer and audio transformation | SPEECH_TO_SPEECH.md |
client.speech_to_text.* |
Audio/video transcription | SPEECH_TO_TEXT.md |
client.audio_isolation.* |
Background noise removal | AUDIO_ISOLATION.md |
client.audio_native.* |
Embeddable audio players | AUDIO_NATIVE.md |
client.forced_alignment.* |
Audio-text timing alignment | FORCED_ALIGNMENT.md |
client.models.* |
Model information and capabilities | MODELS.md |
Configuration Options
Configuration Precedence
- Explicit parameters (highest priority)
- Settings.properties (configured via initializer)
- Environment variables (lowest priority)
Environment Variables
-
ELEVENLABS_API_KEY
- Your ElevenLabs API key (required) -
ELEVENLABS_BASE_URL
- API base URL (optional, defaults tohttps://api.elevenlabs.io
)
Custom Environment Variable Names
client = ElevenlabsClient.new(
api_key_env: "CUSTOM_API_KEY_VAR",
base_url_env: "CUSTOM_BASE_URL_VAR"
)
Error Handling
The client provides specific exception types for different error conditions:
begin
result = client.text_to_speech.convert(voice_id, text)
rescue ElevenlabsClient::AuthenticationError
puts "Invalid API key"
rescue ElevenlabsClient::RateLimitError
puts "Rate limit exceeded"
rescue ElevenlabsClient::ValidationError => e
puts "Invalid parameters: #{e.message}"
rescue ElevenlabsClient::APIError => e
puts "API error: #{e.message}"
end
Exception Types
-
AuthenticationError
- Invalid API key or authentication failure -
RateLimitError
- Rate limit exceeded -
ValidationError
- Invalid request parameters -
NotFoundError
- Resource not found (e.g., voice ID, transcript ID) -
BadRequestError
- Bad request with invalid parameters -
UnprocessableEntityError
- Request cannot be processed (e.g., invalid file format) -
APIError
- General API errors
Rails Integration
The gem is designed to work seamlessly with Rails applications. See the examples directory for complete controller implementations:
- DubsController - Complete dubbing workflow
- TextToSpeechController - TTS with error handling
- StreamingAudioController - Real-time streaming
- TextToDialogueController - Dialogue generation
- SoundGenerationController - Sound effects
- MusicController - AI music composition and streaming
- TextToVoiceController - Voice design and creation
- VoicesController - Voice management (CRUD operations)
- SpeechToSpeechController - Voice changer and audio transformation
- SpeechToTextController - Audio/video transcription with advanced features
- AudioIsolationController - Background noise removal and audio cleanup
- AudioNativeController - Embeddable audio players for websites
- ForcedAlignmentController - Audio-text timing alignment and subtitle generation
Development
After checking out the repo, run:
bin/setup # Install dependencies
bundle exec rspec # Run tests
Available Rake Tasks
# Testing
rake spec # Run all tests (default)
rake test:unit # Run unit tests only
rake test:integration # Run integration tests only
# Code Quality
rake dev:lint # Run RuboCop linter
rake dev:lint_fix # Auto-fix RuboCop issues
rake dev:security # Run security checks
rake dev:audit # Run bundler-audit
# Development
rake dev:test # Run all tests
rake dev:coverage # Run tests with coverage
rake release:prepare # Run full CI suite locally
Continuous Integration
This gem uses GitHub Actions for CI/CD with the following checks:
- Tests: Runs on Ruby 3.0, 3.1, 3.2, and 3.3
- Security: bundler-audit for dependency vulnerability scanning
- Code Quality: RuboCop linting with gem-specific rules
- Build: Verifies gem can be built and installed
All checks must pass before merging pull requests.
To install this gem onto your local machine:
bundle exec rake install
To release a new version:
- Update the version number in
version.rb
- Update
CHANGELOG.md
- Run
bundle exec rake release:prepare
to verify everything passes - Run
bundle exec rake release
Testing
The gem includes comprehensive test coverage with RSpec:
# Run all tests
bundle exec rspec
# Run specific test files
bundle exec rspec spec/elevenlabs_client/endpoints/
bundle exec rspec spec/elevenlabs_client/client
bundle exec rspec spec/integration/
# Run with documentation format
bundle exec rspec --format documentation
Contributing
Bug reports and pull requests are welcome on GitHub at https://github.com/yourusername/elevenlabs_client.
- Fork it
- Create your feature branch (
git checkout -b my-new-feature
) - Commit your changes (
git commit -am 'Add some feature'
) - Push to the branch (
git push origin my-new-feature
) - Create a new Pull Request
License
The gem is available as open source under the terms of the MIT License.
Changelog
See CHANGELOG.md for a detailed list of changes and version history.
Support
- ๐ Documentation: API Documentation
- ๐ Issues: GitHub Issues
- ๐ฌ Discussions: GitHub Discussions
Made with โค๏ธ for the Ruby community