Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is there anyway to properly convert the audio array (float*) into UTF 8? #127

Open
cdrandin opened this issue Mar 24, 2017 · 7 comments
Open

Comments

@cdrandin
Copy link

cdrandin commented Mar 24, 2017

Just wondering if it was possible to somehow turn the array of float values into UTF-8.

I am trying some stuff out that would properly send audio data over a network and play it on another device.

Whenever I try to manipulate the data and receive it on the other end I still end up with floating points.

To put it silly, I kind of need to get it to look like this, bytes

Edit:

More specific what I am trying to do is target a machine that wants to play audio through Port Audio.

@cdrandin
Copy link
Author

Actually, NSData didn't help with translating it over properly. Only popping sounds.

How could I transfer or convert float over properly to be played on another device?

@cdrandin cdrandin reopened this Mar 27, 2017
@alexbw
Copy link
Owner

alexbw commented Mar 27, 2017 via email

@mitisBlack
Copy link

@cdrandin NSData is the answer here, UTF8 is meant for text, so that definitely isn't. Given all the points the conversion could go wrong a small sample or code snippets would go a long way to help anyone point the problem(s).

That being said, it sounds like you are trying to put the data in a string then in NSData and then back. This will probably not work as your data is not valid UTF. Either use a NSArchiver or write data directly to the mutableBytes in NSMutableData

@cdrandin
Copy link
Author

cdrandin commented Apr 7, 2017

@mitisBlack Thanks for the info. I will try those out and see what I end up getting.

This is what I am doing on the iOS side of things.
This was taken from the demo to get mic data.

[self.audioManager setInputBlock:^(float *data, UInt32 numFrames, UInt32 numChannels) {
    // send audio data over
    for (int i = 0; i < (sizeof(data)/sizeof(data[0])); ++i) {
        NSMutableData *convertedData = ...;
        
        [[[WampManager sharedInstance] wampConnection] publishTo:@"com.app.audioStream_ios" payload:@{@"payload": convertedData} result:nil];
    }
}];

Now on the other device side, I use Python that gets the data and tries to play it.

import pyaudio

CHUNK = 512
CHANNELS = 2
RATE = 44100
WIDTH = 2

p = pyaudio.PyAudio()
stream = p.open(format=p.get_format_from_width(WIDTH),
                channels=CHANNELS,
                rate=RATE,
                output=True,
                frames_per_buffer=CHUNK)

def audioStream_ios(payload):
    if stream.is_active():
        # print payload
        self.buffer += payload
        # stream.write(payload)
yield self.subscribe(audioStream_ios, 'com.app.audioStream_ios')

while 1:
    print len(self.buffer)
    if stream.is_active() and len(self.buffer) >= CHUNK * 1:
        data = self.buffer[:CHUNK]
        self.buffer = self.buffer[CHUNK:]
        # print 'write'
        # stream.write(data) # play audio data

    yield sleep(0.01)

EDIT:
So I tried this on the iOS side

    NSKeyedArchiver *ka = [[NSKeyedArchiver alloc] init];
    [ka encodeFloat:data[0] forKey:@"0"];
    [ka encodeFloat:data[1] forKey:@"1"];
     // or with
     NSMutableData *d = [NSMutableData dataWithLength:sizeof(float)];
     [d appendBytes:&data length:sizeof(float)];

     [[[WampManager sharedInstance] wampConnection] publishTo:@"com.app.audioStream_ios" payload:@{@"payload": [ka encodedData]} result:nil];

Got pretty bad feedback still. So, not sure if this is done correctly.

I even try writing the raw PCM values to a file to test it out in Audacity. By importing as raw sound and nothing. signed 16bit PCM, big-endian, sample rate: 44100

 NSData *d = [NSData dataWithBytes:&data length: sizeof(data)];
 [wFileHandle writeData:d];

@mitisBlack
Copy link

@cdrandin I see. I'll take a closer look later, but a quick look over the code tells me you take the wrong size of the incoming data.

sizeof(data)/sizeof(data[0]) returns the size of the pointer (not memory allocated), so ti will probably just result in 2

try using for size numFrames * numChannels

@cdrandin
Copy link
Author

cdrandin commented Apr 11, 2017

@mitisBlack I tried what you recommended.

I am unsure if this is correct, at least how I am storing into a mutable array.

    NSMutableData *d = [NSMutableData dataWithLength:numFrames * numChannels];
    // or     NSData *d = [NSData dataWithBytes:data length: numFrames * numChannels];

    for (int i = 0; i < numFrames * numChannels; i+=1) {
        [d appendBytes:&data[i] length:sizeof(float)];
    }
    
    NSLog(@"%u", numFrames * numChannels); // 1024
    NSLog(@"%lu", (unsigned long)[d length]); // 8192

    
    [[[WampManager sharedInstance] wampConnection] publishTo:@"com.app.audioStream_ios" payload:@{@"payload": d} result:nil];

All comes back as static feedback. I even was playing with the chunk size and the channels on the Python side to see if that helped.

I printed out the hex values of NSData *d = [NSData dataWithBytes: data length: numFrames * numChannels]; in iOS and I printed over what I was getting sent on the Python side with ''.join(x.encode('hex') for x in payload). They resulted in the same hex values on both sides. So, I am unsure where to go from here. The audio works when I use the methods provided by Novocaine library, but once it gets transported or I write to a file myself. It doesn't work.

@cdrandin
Copy link
Author

I have found this example which does a large part of what is needed and I am able to grab the stream. https://github.com/vlider/Mic-recording-minimal-example/blob/master/MicRec/ViewController.swift. Though it is in Swift. I tried writing the raw PCM values directly to file and still came back awful.

So, seems AVAudioEngine may be another choice worth checking out. As I am simply just trying to get it working at this point.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants