Newbie friendly, DRM-only opengles, /init tutorial

[habemus-papadum]

Hi Miouyouyou!

I was wondering if you would consider the following idea for a tutorial at some point; it is a followup from my comments at your gist:

Create a wiki page detailing the steps required to build a minimal system image that boots and runs your opengles example as an /init script. From some of your writings, I think you have an interest in android; this tutorial would have a slightly different target audience: people interested creating absolute minimal devices in which they can control and understand nearly every part of the process and nevertheless can access the full power of the cpu/gpu (e.g. have proper Mali drivers). I'm not sure how interesting this would be for you, but if you have questions I can try to flesh out more details and provide better explanations. (and as you pointed out in your comment on the gist, the remaining black box/proprietary stuff would be the Mali drivers themselves, which for me is something I can live with....)

For the time being, I was thinking: Since most of the complicated details are worked out (RockMyy, the opengl gist, etc), it might be good to make the tutorial newbie friendly:

• starting with some basic advise on selecting a board
• assembling a tool chain (keep this as simple as possible...)
• building RockMyy -- menuconfig options, laying out root file system
• getting mali drivers from rockchip, creating symlinks
• building opengl app
• creating image and creating sd card

If you are interested we could do a collaboration where you document the hard parts and I document the newbie parts.

Thanks!

[Miouyouyou]

Greetings @habemus-papadum ! 😃

Yeah, I'll try to put in place several tutorials concerning the installation, and the use of the Mali user-space binary drivers with :

• Raw OpenGL ES with the EGL + DRM/KMS/GBM backend
• SDL 2.0.6 + DRM backend, if I can make it work
  And a few scripts to install these drivers quickly, since installing them is quite a pain.

I also have to document how to use the cross-compilation script. Though, the script is now set up to use the default cross-compiler installed on Debian/Ubuntu systems when you do :
apt install crossbuild-essential-armhf
So, if you do that, you can just grab the script, run it with bash : bash ./GetPatchAndCompileKernel.sh, wait for its completion and you should have a /tmp/RockMyy-Build folder with a freshly cross-compiled kernel in it, that you can run on your board.

The hardest part being how to copy the files on your system and boot the new kernel. Generally this involves creating a tarball and send it to your board through SSH, using scp :

export YOUR_BOARD_IP=10.100.0.55
cd /tmp/RockMyy-Build
tar cJpvf kernel.tar.xz *
scp kernel.tar.xz $YOUR_BOARD_IP:/tmp
ssh $YOUR_BOARD_IP

And then install the files in the tarball, from the board :

# On the board, through SSH
cd /tmp
tar Jxpvf kernel.tar.xz
cp -r lib/* /lib/
cp -r usr/* /usr/

Then, for the boot/ files, you'll have to understand how your bootloader is setup. With my setup, I only do :

cp boot/* /boot

But that only works because I have an extlinux.conf that boots /boot/zImage and uses the dtb /boot/rk3288-miqi.dtb . If you look at the thread I linked, on the second page, I've documented my setup. That should be a good base for a documentation.

Now my setup is generally incompatible with U-boot scripts, so the user looking to install and test that new kernel will have to tinker the boot files installations depending on the bootloader configuration of its distribution, or forget about kernel updates provided by its distribution.
The best way to document this is to understand how the most popular distributions running on RK3288 (Armbian, TinkerOS, Elar Systems images, ...) are setup and provide a few appropriate commands to install the boot files in each case.
A better way would to provide an installation script, or prepackaged files, but that requires a lot of trial and error.

Now, any user wishing to do install a new custom kernel will have to understand how to get a serial connection to the board, and use it to debug issues when the board does not boot anymore. Because that will happen and is generally a source of panic for many users. So better understand how to do that BEFORE attempting to install new kernels that way.

For the board, at the moment, I can only recommend using a MiQi, since it's the only board I own and can test thoroughly. I'll try to get my hands on other boards. The Armbian team tried to get some ASUS Tinkerboard samples for me, but ASUS never replied to their requests 😕 . Also, this board requires patches to reboot the system correctly so... yeah...
If I can get my hands on a Firefly-RK3288, I'll try to test it, as that seems to be a quite solid system.

Meanwhile, I've never tested the SDCard boot. It might work on a MiQi but I frankly don't how it will fare.

Anyway, yeah, I'm interested in building a documentation and good for collaboration, as long as the documentation stays CC0 or CC-BY 🐱

[habemus-papadum]

@Miouyouyou
thanks for your comments.

I was thinking a little more about this, and here are my latest thoughts
toolchain: my linux box runs centos which never has any useful packages, but in any case I was thinking of using the linaro aarch64 tool chain

linaro tool chain comes with a sysroot, and works well enough on my centos box, and for this project I don't mind being lazy...

so the steps would be:

• userspace
  • copy mali userspace drivers into linaro sysroot
  • build opengl app using linaro tool chain
  • collect app and dependent shared libs into a root file system for the device (around 4-6 files total: app, ld-linux, libc, mali libs)
• kernel
  build with rockmyy script and linaro toolchain, but use menuconfig to properly configure initrd using the root file system from above

this bzImage is fully self contained and ready to go

some questions I'm not sure about --

• simplest way to boot kernel? as you said this seems hard. current choices are:
  • for tinkerboard, modify the uboot scripts used by tinkeros.
  • using your suggestions, scp into armian/tinkeros device and add another kernel -- i'm not quite sure about all the details
• how to know for sure that 3d graphics will go to hdmi port (highly likely, but not positive)? also, as you said, work out details for serial port during boot

I plan to play around with things on my end and keep posting notes here. on my todo:

• check that using linaro toolchain to build kernel and user space app works fine (hopefully this is easy)
• using existing distros, fig
  ure out details of uboot and also how to debug kernel boot over serial port
• combine everything together.
  (I have a tinkerboard)
  Sp maybe we will both just try things out until we figure out a workflow that is "easy" and then try to right up the results?

[Miouyouyou]

Toolchain

As long as the aarch64 toolchain can compile armv7 (32 bits) executables, it's fine. Else, you'll have to also get the usual armv7 (generally named "arm") toolchain from linaro too.

Compilation of user-space programs

Now, you don't really need to copy the Mali libraries to compile projects. This is nice. But you can also use the Mesa ones, if these are easier to obtain. You could even put placeholders libraries containing only the "right" symbols names with no actual code in them.

Note that when you compile, the main purpose of specifying the libraries names is to generate a list of files that will be put in the ELF executable.
Very roughly, when executed the ELF binary will be put into memory and the dynamic linker will be loaded. The address of the dynamic linker's dynamic symbol lookup function will be written as the address of every dynamic symbol in the in-memory ELF file so that, when the address of a dynamic symbol is currently unknown and must be looked up (dynamic symbol being generally the function name available in a dynamic library. i.e. : glCreateShader), the dynamic linker dynamic symbol lookup function will be executed (generally it's stored in /lib/ld.so.something on linux systems and another one on Android systems).
When executed, the dynamic linker dynamic symbol lookup function will load in memory the dynamic libraries put in the ELF executable libraries list, if not done already, and search for the asked symbol in each loaded library until it finds it. Then it will write the address back into in-memory ELF file so that future calls of the same dynamic symbol lead to a direct jump to the now "resolved" address.

The whole point of that diatribe is to understand that, the libraries during the compilation and the libraries during the execution can be entirely different, as long as they share the same symbols names.
What really matter is that the system executing the binary has the same libraries names with the same symbols names, and appropriate executable code in these libraries..
The ELF file only contains libraries "names". It doesn't care about the content of these libraries during the compilation.
During the compilation, the library can full of empty functions. As long that it has the right symbols (because these will be checked by GCC, just to be sure you're talking about the right library), this will compile.

So, you can compile against Mesa libraries and execute with Mali libraries and everything should be fine. Of course, if you can use the Mali ones for both, this will avoid any potential symbol mismatch issue.

Kernel compilation

Now, I wonder if you can't specify an initrd file to use directly through environment variables interpreted during the kernel compilation itself (Meaning, through environment variables setup by the kernel developers themselves).
Anyway, you'll need to execute the script like this :

MAKE_CONFIG=menuconfig bash ./GetPatchAndCompileKernel.sh

To get access to menuconfig when compiling with my script.

Booting

Now, for the boot part... it's always an issue when beginning.
I'll have to play with different U-Boot configurations on different distributions to provide useful hints.
I'll also repaste my extlinux.conf setup tomorrow.

Documentation

Anyway, yeah, this will require some trial and error and a documentation, or at least some notes, of these "trials", "errors" and "solutions". Documentation of the potential errors is required to avoid letting new comers in a broken state, which lead people to panic and/or disgust.

[habemus-papadum]

aarch64 -> uh, yeah, I was spacing out, sorry. Will use 32 bit toolchain

the main purpose of specifying the libraries names

fun fact, Apple no longer provides any binaries in the sysroots for all its devices it bundles with Xcode. instead there is a yaml file for each shared lib listing symbols names etc that the system linker knows how to process (each sysroot is still several 100MB...)

I thought to use the libmali drivers because it seemed like a pain to to install mesa! -- too big, too many pieces that I don't need (X support, proper opengl, etc) -- Copying over a few libmali libs and headers and wrangling a few symlinks felt much easier... Funny how people see the same thing in almost opposite terms... 😄

initrd -> I don't know about env variables, but I was going to menuconfig because from there you can form the initrd in two parts -- 1) a 'script' which can do things like properly layout /dev devices (/dev/null, etc) without having to call sudo, and 2) path to a local dir (containing app and shared libs) which will be cpio'd into one nice package

booting -- ah, booting -- that might be a pain...

I should start recording steps sometime next week.

[Miouyouyou]

fun fact, Apple no longer provides any binaries in the sysroots for all its devices it bundles with Xcode. instead there is a yaml file for each shared lib listing symbols names etc that the system linker knows how to process (each sysroot is still several 100MB...)

Well, that makes things clearer and easier to maintain 😃.

Yeah, using the mali drivers and just creating symlinks can be way faster than getting the whole Mesa artillery, indeed. Just wanted to clear some potential misconceptions.

Concerning the /boot/extlinux/extlinux.conf, it's defined like this :

label kernel-4.4
    kernel /boot/zImage
    fdt /boot/rk3288-miqi.dtb
    append  earlyprintk console=ttyS2,115200n8 rw root=/dev/mmcblk1p1 rootfstype=ext4 init=/sbin/init

It's a pretty standard extlinux.conf. As long as the right zImage and dtb files are copied in /boot, it should boot just fine.
Note that I do not have any boot.cmd or boot.scr file in the /boot folder. I think U-boot will parse these files first THEN the extlinux/extlinux.conf.
So, yeah, I guess that adding an initrd line with the path to the initrd image will do the trick.

I'll give it a try this week-end.

[habemus-papadum]

Cool -- So I have very little knowledge about uboot/extlinux.conf at this point.

but my basic plan for tinkerboard was to use a variant of the steps here to create a uboot image ont which I would add your extlinux.conf from above (with the tinker.dtb and kernel from this repo)

hopefully it will work!

[habemus-papadum]

also -- to be clear, my intention is to combine the initrd into the zImage as well as compile all modules statically into the kernel, so no initrd entry on the boot command line -- I guess this would be "classic, embedded style" as opposed to what distros do to provide the possibility for varied hardware and easy upgrade of the kernel... I don't know if this is possible with the dtb as well, but if it is, I will bundle that into the zImage as well

[Miouyouyou]

It should be possible to include everything inside the kernel. That basically what I was doing at first, until a few people asked me to include support for various hardware. Beside a few modules (mali_kbase and some Rockchip HDMI CEC modules), you can remove most the modules without any problem.

Note that some "staging" modules actually include the WiFi driver of the Tinkerboard though.

[habemus-papadum]

Note that some "staging" modules actually include the WiFi driver of the Tinkerboard though.

I'm not following... what are "staging modules"? In any case I was thinking of leaving all hardware support enabled, but just statically built. that way the final work flow is that you build the kernel once. and then when you iterate your app, you build it and then have kbuild redo the ziImage bunding, and deploy to device (initially via flash, but via tftp asap)

[habemus-papadum]

What happens when a touch screen is added to the board? (I don't have one). Does it show up as a second monitor or is video mirrored between the hdmi and touchscreen? Does the egl demo need to be updated to support users that may have different video configurations (touch screen only, etc)?

[Miouyouyou]

@Tonymac32 tested a touch screen on a Tinkerboard, and it seemed to act as a secondary screen at first. However, I'm pretty sure that you can configure how your display system (X11 or DRM) uses the screen afterwards.

If you intend to choose the screen (crtc + encoder) on which you want to display the demo, yes, you'll have to modify the EGL demo. More importantly, you'll have to change the DRM part of the demo. EGL doesn't really care about screens anyway.

Concerning the "staging" modules, there is a "staging drivers" section in the kernel that contain drivers which are integrated but are terribly written. The Realtek Wifi driver used by the Tinkerboard is currenty in that "staging drivers" section, so take a look at that section if you're looking for the WiFi drivers of your Tinkerboard.

For input support, you'll have to deal either evdev (low level), libinput (a bit higher level, using evdev) or maybe SDL if it works for you. The last one being more interesting if you intend to use Gamepads afterwards.

[Miouyouyou]

Here's the thread about touch screen tests : https://forum.armbian.com/topic/4793-tinker-board-and-the-rpi-7-touchscreen/

[habemus-papadum]

Ok --- So I'm hoping to start working on things during the down time of the holidays.

A couple of questions:

• So it look's like you now have a tinkerboard? Are you still waiting for SD cards or have you had a chance to boot it? In the past, I think the workflow you use is start with a third party distribution and then replace the kernel with the RockMyy-build, tweaking the bootloader etc as needed. For the tinkerboard, will you use TinkerOS or armbian as the starting point? In either case, was there anything tricky required to have the boot loader see your kernel?

• There seems to be some hacki-ness related to the tinkerboard not rebooting properly -- other than that are there any known limitations with the tinkerboard?

• Serial port: I have not yet sorted out how I will connect to the device to see boot logs etc. Do you use the UART via GPIO pins for serial access or is it serial over USB (i guess I'm asking for either MIQI or tinkerboard) It seems like USB would be more convenient, but the GPIO way might be more foolproof. If GPIO, did you just make your own cable or is there some stock cable that you would recommend. How does one go about figuring out the name of the serial port to pass to the kernel during boot?

• So I was thinking of trying to rebuild your kernel with all 'm''s changed to 'y''s in the kernel config... Off the top of your head do you see any issues I might run into if I do this?

thanks!

[Miouyouyou]

Hey !

• I'm still waiting for some SDCard to arrive before testing the Tinkerboard. I'll try ARMbian at the moment, in order to be "on-par" with the MiQi. However, since it works with MicroSD cards, I might be able to generate one with TinkerOS and see how it fares.

• The Tinkerboard has several issues regarding reboot, since the MMC hardware gets disabled during the reboot phase and it seems that a software reboot can't enable the MMC hardware back. I'm pretty sure the Rockchip chip guys have a special reboot mode that triggers a real power cycle, instead of a "soft" reboot and that reboot should be used but... Since I didn't have any Tinkerboard to test that idea, I just ported the patch provided by the ASUS team on their 4.4 kernel.
  Still, fixing this issue was a real pain to fix : 4.12.0-rc4 Kernel: Reboot does not work MyyQi#8
  Note, that the Bluetooth driver also is not ported. There are a few bits in "staging" section of the Linux kernel that might help get the chip working, but I haven't been able to get these work at the moment.

• Concerning UART, I'm currently using a USB to UART (pins) cable so I guess it's a USB to GPIO pins. Not absolutely sure but it's clearly not connected to the USB ports of the MiQi, that's for sure.
  Now, the serial output are described in the RK3288 reference manuals. The one I currently use to debug my MiQi when serious issues happen is /dev/ttyS2 . But deducing the port name was more a bit of trial and error than anything.

• Right off the bat, I don't remember any issue that could happen if everything is built inside the kernel, instead of building things as modules. I'll have to check my Git logs though.

[habemus-papadum]

Thanks -- all very good to know!

…

On Sun, Dec 17, 2017 at 1:53 PM, Miouyouyou ***@***.***> wrote: Hey ! - I'm still waiting for some SDCard to arrive before testing the Tinkerboard. I'll try ARMbian at the moment, in order to be "on-par" with the MiQi. However, since it works with MicroSD cards, I might be able to generate one with TinkerOS and see how it fares. - The Tinkerboard has several issues regarding reboot, since the MMC hardware gets disabled during the reboot phase and it seems that a software reboot can't enable the MMC hardware back. I'm pretty sure the Rockchip chip guys have a special reboot mode that triggers a real power cycle, instead of a "soft" reboot and that reboot should be used but... Since I didn't have any Tinkerboard to test that idea, I just ported the patch provided by the ASUS team on their 4.4 kernel. Still, fixing this issue was a real pain to fix : Miouyouyou/MyyQi#8 <Miouyouyou/MyyQi#8> Note, that the Bluetooth driver also is not ported. There are a few bits in "staging" section that might help get the chip working, but I haven't been able to get this work at the moment. - Concerning UART, I'm currently using a USB to UART (pins) cable so I guess it's a USB to GPIO pins. Not absolutely sure but it's clearly not connected to the USB ports of the MiQi, that's for sure. Now, the serial output are described in the RK3288 reference manuals. The one I currently use to debug my MiQi when serious issues happen is /dev/ttyS2 . But deducing the port name was more a bit of trial and error than anything. - Right off the bat, I don't remember any issue that could happen if everything is built inside the kernel, instead of building things as modules. I'll have to check my Git logs though. — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#2 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABG5hK8WLQEyC9-XH8aFIYo4QDTTF_ITks5tBWMWgaJpZM4QUveu> .

[habemus-papadum]

Hi -- Happy new year!

So, still not made much progress on this on my end -- no real good excuse. I did work on it a little during the holidays; it's not much but here is what I did and noticed:

• So I got my tinker hardware running and confirmed that it boots fine with the stock tinkeros image.
• I purchased USB serial cable from Adafruit and tweaked boot params to make sure I could see kernel boot over serial

---> So now I have a sane dev enviroment -- yeah!

• next, another sanity check, I rebuilt the kernel bundled with tinkeros using the instructions at
  https://tinkerboarding.co.uk/wiki/index.php?title=Software#Kernel
• Their kernel is 4.4 and uses older Mali drivers...
• Once I did this, I wanted to investigate tweaking their kernel to have no kernel modules loaded at runtime (I'm not sure why I am against loading modules at runtime, but for some reason I am 😄 )
• It turns out in their config that almost all of the drivers are statically linked into the kernel -- which is nice. There was one exception (either wifi or bluetooth, I don't remember)
• I tried modifying their kernel config so that this driver would also be statically linked, but it turns out their are multiple modules being controlled by one config option and if you try to statically link them all into the kernel you get link errors due to duplicate symbols.

At this point I stopped with the tinkeros kernel and started looking into yours.

• I had no problems building the kernel, but noticed that by default most of your drivers are loaded at runtime. It seemed like it might be a big project to convert them to static linking because again there might be symbol collisions.

It's not clear to me that trying to statically link in drivers into the kernel is that important. Aesthetically it felt important, but I don't know how much complexity it would add (right now, your one kernel config supports many boards, but switching to static drivers would mean more complexity.)

Still I thought I would be worthwhile to try to carefully diff tinker's kernel config vs yours -- it would be educational for me if nothing else -- but time ran out and I haven't had a chance.

Stupidly, I did not get around to just trying out the opengl stuff.

Cheers, nehal

[Miouyouyou]

The first kernels I cross-compiled for MiQi devices had almost everything in static. There were still a few modules for USB Wifi dongles and NFS support, but that's it.

Then I had requests to support more and more things "Out-of-the-box", so I added the modules but it should not be too hard to compile the strict necessary statically.
Now, the Wifi/Bluetooth driver used by the Tinkerboard being a "staging" driver, it might be the biggest issue at the moment. Also, I think there's an issue with some Mali drivers configuration, indeed.

Try to remove unwanted features (Like DVB, webcams, USB network adapters support, ...) and compile the rest as static and see how it fares.
If there's an error, provide the ~30 last lines that talk about the error, I'll see if I can provide a solution.

I'll still try to compile everything in static this week-end and see how it fares.

[habemus-papadum]

Thanks -- makes sense that you transitioned to modules. i'll let you know how it goes. nehal

[tiandaozhang]

Hi Miouyouyou, I compile the kernel 4.15-rc8 for MIQI . I change the device tree for rk3288-miqi.dts for adding I2S and SPDIF. The board can start up normal, but no sound to play.
The log for spdif is below:
root@linaro-alip:~# aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: SPDIF [SPDIF], device 0: ff8b0000.sound-dit-hifi dit-hifi-0 []
Subdevices: 1/1
Subdevice #0: subdevice #0

root@linaro-alip:~# aplay 250Hz_44100Hz_16bit_30sec.wav
Playing WAVE '250Hz_44100Hz_16bit_30sec.wav' : Signed 16 bit Little Endian, Rate 44100 Hz, Mono

The SPDIF port is normal. I had tested it in other ways. Now I don't know how to test it. Do you have any suggestions? Thank you.

[Miouyouyou]

Greetings @tiandaozhang,

Quite recently, @Tonymac32 was able to get some nice sound output on its Tinkerboard and provided me some instructions on how to get the sound output working on these boards.
That said, I haven't tested them yet and I haven't tried on the MiQi.

Still, you can give it a try and see how it fares.

Here's the configuration files you'll have to modify and the modifications you'll have to bring.

/etc/asound.conf

pcm.OnBoard_D0 {
        type hw
        card OnBoard
        device 0
}

pcm.OnBoard_D1 {
        type hw
        card OnBoard
        device 1
}

pcm.OnBoard_D2 {
        type hw
        card OnBoard
        device 2
}

/etc/pulse/default.pa

#load-module module-alsa-sink

load-module module-alsa-sink device=OnBoard_D0 sink_properties=device.description="BT_VOIP-Output"
load-module module-alsa-sink device=OnBoard_D1 sink_properties=device.description="SPDIF-Output"
load-module module-alsa-sink device=OnBoard_D2 sink_properties=device.description="Headset-Output"
#load-module module-alsa-source device=hw:1,0
load-module module-alsa-source device=OnBoard_D0 source_properties=device.description="BT_VOIP-Input"
load-module module-alsa-source device=OnBoard_D1 source_properties=device.description="Headset-Input"

[tiandaozhang]

Thanks for you help.
I add asound.conf and default.pa to my system, but it still can not broadcast sound on the spdif port. I use oscilloscope to test spdif_tx port, but there is not signal on it.

The dts file
`
RockMyy/linux/arch/arm/boot/dts/rk3288-miqi.dts

#include <dt-bindings/input/input.h>
#include "rk3288.dtsi"

/ {
model = "mqmaker MiQi";
compatible = "mqmaker,miqi", "rockchip,rk3288";

    chosen {
            stdout-path = "serial2:115200n8";
    };

    memory@0 {
            device_type = "memory";
            reg = <0x0 0x0 0x0 0x80000000>;
    };


    sound {
            compatible = "simple-audio-card";
            simple-audio-card,name = "SPDIF";

            simple-audio-card,dai-link@1 {  /* S/PDIF - S/PDIF */
                    cpu {
                            sound-dai = <&spdif>;
                    };
                    codec {
                            //sound-dai = <&rockchip_spdif>;
                            sound-dai = <&spdif_out>;
                    };
            };
                            /*
             * If you want to support more cards,
             * you can add more dai-link node,
             * such as
             *
             *   simple-audio-card,dai-link@1 {
             *       ......
             *   }
             */

    };

    //rockchip_spdif: rockchip-spdif {
    spdif_out: spdif-out {
            compatible = "linux,spdif-dit";
            #sound-dai-cells = <0>;
    };

......

&i2s {
#sound-dai-cells = <0>;
status = "okay";
};

&spdif {
//#sound-dai-cells = <0>;
status = "okay";
};

.....`

[Miouyouyou]

Anything in dmesg regarding SPDIF ?

I'm looking at that SPDIF driver, spdif-dit, but it's quite light... Are you driving the SPDIF through GPIO or through the HDMI interface ?

[Miouyouyou]

Also did you try with the sound-dai = <&rockchip_spdif>; codec ?

[tiandaozhang]

Yes , I had try sound-dai = <&rockchip_spdif>.

[tiandaozhang]

The dmesg log is below:

dmesg_180125.zip

[Miouyouyou]

Could you retry with the rockchip_spdif one more time and provide the dmesg output ?

[tiandaozhang]

OK

dmesg_rockchip_spdif.zip

           simple-audio-card,dai-link@1 {  /* S/PDIF - S/PDIF */
                    cpu {
                            sound-dai = <&spdif>;
                    };
                    codec {
                            sound-dai = <&rockchip_spdif>;
                            //sound-dai = <&spdif_out>;
                    };
            };
    };

    rockchip_spdif: rockchip-spdif {
    //spdif_out: spdif-out {
            compatible = "linux,spdif-dit";
            #sound-dai-cells = <0>;
    };

[tiandaozhang]

Almost there isn't difference between the two log files about SPDIF.

[Miouyouyou]

Could you try to also replace linux,spdif-dit by rockchip,rk3228-spdif ?

[tiandaozhang]

I have test it , and there isn't sound card found in this way.
spdif_out: spdif-out {
//compatible = "linux,spdif-dit";
compatible = "rockchip,rk3228-spdif";
#sound-dai-cells = <0>;
}
test_rockchip_spdif.zip

[tiandaozhang]

In order to output audio to hdmi and I2S(GND,I2S_SCLK,I2S_LRCK_TX,I2S_SDO0) at the same time. When I play sound, there isn't data in I2S port. I don't know why? @Miouyouyou

[Miouyouyou]

I'll try to see how sound work on MiQi devices this evening.

Meanwhile, try to do this instead, for the sound output :

        sound {
                compatible = "simple-audio-card";
                simple-audio-card,format = "i2s";
                simple-audio-card,name = "rockchip,tinker-codec";
                simple-audio-card,mclk-fs = <512>;

                simple-audio-card,codec {
                        sound-dai = <&hdmi>;
                };

                simple-audio-card,cpu {
                        sound-dai = <&i2s>;
                };
        };

Remove the spdif_out nodes but keep the i2s node.
You can also keep the spdif node, but I think it's only useful when you have an actual SPDIF output, like on Firefly devices.

Example rk3288-miqi.dts in this case.

/*
 * Copyright (c) 2016 Heiko Stuebner <heiko@sntech.de>
 *
 * This file is dual-licensed: you can use it either under the terms
 * of the GPL or the X11 license, at your option. Note that this dual
 * licensing only applies to this file, and not this project as a
 * whole.
 *
 *  a) This file is free software; you can redistribute it and/or
 *     modify it under the terms of the GNU General Public License as
 *     published by the Free Software Foundation; either version 2 of the
 *     License, or (at your option) any later version.
 *
 *     This file is distributed in the hope that it will be useful,
 *     but WITHOUT ANY WARRANTY; without even the implied warranty of
 *     MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 *     GNU General Public License for more details.
 *
 * Or, alternatively,
 *
 *  b) Permission is hereby granted, free of charge, to any person
 *     obtaining a copy of this software and associated documentation
 *     files (the "Software"), to deal in the Software without
 *     restriction, including without limitation the rights to use,
 *     copy, modify, merge, publish, distribute, sublicense, and/or
 *     sell copies of the Software, and to permit persons to whom the
 *     Software is furnished to do so, subject to the following
 *     conditions:
 *
 *     The above copyright notice and this permission notice shall be
 *     included in all copies or substantial portions of the Software.
 *
 *     THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
 *     EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
 *     OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
 *     NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
 *     HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
 *     WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
 *     FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
 *     OTHER DEALINGS IN THE SOFTWARE.
 */

/dts-v1/;
#include <dt-bindings/input/input.h>
#include "rk3288.dtsi"

/ {
	model = "mqmaker MiQi";
	compatible = "mqmaker,miqi", "rockchip,rk3288";

	chosen {
		stdout-path = "serial2:115200n8";
	};

	memory@0 {
		device_type = "memory";
		reg = <0x0 0x0 0x0 0x80000000>;
	};

	ext_gmac: external-gmac-clock {
		compatible = "fixed-clock";
		#clock-cells = <0>;
		clock-frequency = <125000000>;
		clock-output-names = "ext_gmac";
	};

	leds {
		compatible = "gpio-leds";

		fan {
			gpios = <&gpio0 RK_PC2 GPIO_ACTIVE_HIGH>;
			label = "miqi:green:fan";
			linux,default-trigger = "heartbeat";
		};


		work {
			gpios = <&gpio7 RK_PA2 GPIO_ACTIVE_HIGH>;
			label = "miqi:green:user";
			linux,default-trigger = "timer";
		};
	};

	sound {
		compatible = "simple-audio-card";
		simple-audio-card,format = "i2s";
		simple-audio-card,name = "rockchip,tinker-codec";
		simple-audio-card,mclk-fs = <512>;

		simple-audio-card,codec {
			sound-dai = <&hdmi>;
		};

		simple-audio-card,cpu {
			sound-dai = <&i2s>;
		};
	};

	vcc_flash: flash-regulator {
		compatible = "regulator-fixed";
		regulator-name = "vcc_flash";
		regulator-min-microvolt = <1800000>;
		regulator-max-microvolt = <1800000>;
		vin-supply = <&vcc_io>;
	};

	vcc_host: usb-host-regulator {
		compatible = "regulator-fixed";
		enable-active-high;
		gpio = <&gpio0 RK_PB6 GPIO_ACTIVE_HIGH>;
		pinctrl-names = "default";
		pinctrl-0 = <&host_vbus_drv>;
		regulator-name = "vcc_host";
		regulator-min-microvolt = <5000000>;
		regulator-max-microvolt = <5000000>;
		regulator-always-on;
		vin-supply = <&vcc_sys>;
	};

	vcc_sd: sdmmc-regulator {
		compatible = "regulator-fixed";
		gpio = <&gpio7 RK_PB3 GPIO_ACTIVE_LOW>;
		pinctrl-names = "default";
		pinctrl-0 = <&sdmmc_pwr>;
		regulator-name = "vcc_sd";
		regulator-min-microvolt = <3300000>;
		regulator-max-microvolt = <3300000>;
		startup-delay-us = <100000>;
		vin-supply = <&vcc_io>;
	};

	vcc_sys: vsys-regulator {
		compatible = "regulator-fixed";
		regulator-name = "vcc_sys";
		regulator-min-microvolt = <5000000>;
		regulator-max-microvolt = <5000000>;
		regulator-always-on;
		regulator-boot-on;
	};


	cpu0_opp_table: opp_table {
		compatible = "operating-points-v2";
		opp-shared;

		opp@600000000 {
			opp-hz = /bits/ 64 <600000000>;
			opp-microvolt = <900000>;
		};
		opp@816000000 {
			opp-hz = /bits/ 64 <816000000>;
			opp-microvolt = <1000000>;
		};
		opp@1008000000 {
			opp-hz = /bits/ 64 <1008000000>;
			opp-microvolt = <1050000>;
		};
		opp@1200000000 {
			opp-hz = /bits/ 64 <1200000000>;
			opp-microvolt = <1100000>;
		};
		opp@1416000000 {
			opp-hz = /bits/ 64 <1416000000>;
			opp-microvolt = <1200000>;
		};
		opp@1512000000 {
			opp-hz = /bits/ 64 <1512000000>;
			opp-microvolt = <1250000>;
		};
		opp@1608000000 {
			opp-hz = /bits/ 64 <1608000000>;
			opp-microvolt = <1300000>;
		};
		opp@1704000000 {
			opp-hz = /bits/ 64 <1704000000>;
			opp-microvolt = <1350000>;
		};
		opp@1800000000 {
			opp-hz = /bits/ 64 <1800000000>;
			opp-microvolt = <1400000>;
		};
		/* boot-only frequencies below */
		opp@1896000000 {
			opp-hz = /bits/ 64 <1896000000>;
			opp-microvolt = <1425000>;
			turbo-mode;
		};
		opp@1920000000 {
			opp-hz = /bits/ 64 <1920000000>;
			opp-microvolt = <1425000>;
			turbo-mode;
		};
		opp@1992000000 {
			opp-hz = /bits/ 64 <1992000000>;
			opp-microvolt = <1450000>;
			turbo-mode;
		};
		opp@2016000000 {
			opp-hz = /bits/ 64 <2016000000>;
			opp-microvolt = <1475000>;
			turbo-mode;
		};
		opp@2040000000 {
			opp-hz = /bits/ 64 <2040000000>;
			opp-microvolt = <1475000>;
			turbo-mode;
		};
		opp@2064000000 {
			opp-hz = /bits/ 64 <2064000000>;
			opp-microvolt = <1475000>;
			turbo-mode;
		};
		opp@2088000000 {
			opp-hz = /bits/ 64 <2088000000>;
			opp-microvolt = <1500000>;
			turbo-mode;
		};
		opp@2112000000 {
			opp-hz = /bits/ 64 <2112000000>;
			opp-microvolt = <1500000>;
			turbo-mode;
		};
		opp@2136000000 {
			opp-hz = /bits/ 64 <2136000000>;
			opp-microvolt = <1500000>;
			turbo-mode;
		};
		opp@2160000000 {
			opp-hz = /bits/ 64 <2160000000>;
			opp-microvolt = <1500000>;
			turbo-mode;
		};
		opp@2184000000 {
			opp-hz = /bits/ 64 <2184000000>;
			opp-microvolt = <1500000>;
			turbo-mode;
		};
		opp@2208000000 {
			opp-hz = /bits/ 64 <2208000000>;
			opp-microvolt = <1500000>;
			turbo-mode;
		};
	};
};
 
&cpu0 {
	cpu0-supply = <&vdd_cpu>;

	operating-points-v2 = <&cpu0_opp_table>;
};

&emmc {
	bus-width = <8>;
	cap-mmc-highspeed;
	disable-wp;
	non-removable;
	pinctrl-names = "default";
	pinctrl-0 = <&emmc_clk>, <&emmc_cmd>, <&emmc_pwr>, <&emmc_bus8>;
	vmmc-supply = <&vcc_io>;
	vqmmc-supply = <&vcc_flash>;
	status = "okay";
};

&gmac {
	assigned-clocks = <&cru SCLK_MAC>;
	assigned-clock-parents = <&ext_gmac>;
	clock_in_out = "input";
	pinctrl-names = "default";
	pinctrl-0 = <&rgmii_pins>, <&phy_rst>, <&phy_pmeb>, <&phy_int>;
	phy-supply = <&vcc_lan>;
	phy-mode = "rgmii";
	snps,reset-active-low;
	snps,reset-delays-us = <0 10000 1000000>;
	snps,reset-gpio = <&gpio4 RK_PB0 GPIO_ACTIVE_LOW>;
	tx_delay = <0x30>;
	rx_delay = <0x10>;
	status = "ok";
};

&gpu {
	mali-supply = <&vdd_gpu>;
	status = "okay";
};

&hdmi {
	ddc-i2c-bus = <&i2c5>;
	status = "okay";
};

&i2c0 {
	clock-frequency = <400000>;
	status = "okay";

	vdd_cpu: syr827@40 {
		compatible = "silergy,syr827";
		fcs,suspend-voltage-selector = <1>;
		reg = <0x40>;
		regulator-name = "vdd_cpu";
		regulator-min-microvolt = <712500>;
		regulator-max-microvolt = <1500000>;
		regulator-always-on;
		regulator-boot-on;
		regulator-enable-ramp-delay = <300>;
		regulator-ramp-delay = <8000>;
		vin-supply = <&vcc_sys>;
	};

	vdd_gpu: syr828@41 {
		compatible = "silergy,syr828";
		fcs,suspend-voltage-selector = <1>;
		reg = <0x41>;
		regulator-name = "vdd_gpu";
		regulator-min-microvolt = <712500>;
		regulator-max-microvolt = <1500000>;
		regulator-always-on;
		vin-supply = <&vcc_sys>;
	};

	hym8563: hym8563@51 {
		compatible = "haoyu,hym8563";
		reg = <0x51>;
		#clock-cells = <0>;
		clock-frequency = <32768>;
		clock-output-names = "xin32k";
	};

	act8846: act8846@5a {
		compatible = "active-semi,act8846";
		reg = <0x5a>;
		pinctrl-names = "default";
		pinctrl-0 = <&pmic_vsel>;
		system-power-controller;

		vp1-supply = <&vcc_sys>;
		vp2-supply = <&vcc_sys>;
		vp3-supply = <&vcc_sys>;
		vp4-supply = <&vcc_sys>;
		inl1-supply = <&vcc_sys>;
		inl2-supply = <&vcc_sys>;
		inl3-supply = <&vcc_20>;

		regulators {
			vcc_ddr: REG1 {
				regulator-name = "vcc_ddr";
				regulator-always-on;
			};

			vcc_io: REG2 {
				regulator-name = "vcc_io";
				regulator-min-microvolt = <3300000>;
				regulator-max-microvolt = <3300000>;
				regulator-always-on;
			};

			vdd_log: REG3 {
				regulator-name = "vdd_log";
				regulator-min-microvolt = <1100000>;
				regulator-max-microvolt = <1100000>;
				regulator-always-on;
			};

			vcc_20: REG4 {
				regulator-name = "vcc_20";
				regulator-min-microvolt = <2000000>;
				regulator-max-microvolt = <2000000>;
				regulator-always-on;
			};

			vccio_sd: REG5 {
				regulator-name = "vccio_sd";
				regulator-min-microvolt = <3300000>;
				regulator-max-microvolt = <3300000>;
				regulator-always-on;
			};

			vdd10_lcd: REG6 {
				regulator-name = "vdd10_lcd";
				regulator-min-microvolt = <1000000>;
				regulator-max-microvolt = <1000000>;
				regulator-always-on;
			};

			vcca_18: REG7 {
				regulator-name = "vcca_18";
				regulator-min-microvolt = <1800000>;
				regulator-max-microvolt = <1800000>;
			};

			vcca_33: REG8 {
				regulator-name = "vcca_33";
				regulator-min-microvolt = <3300000>;
				regulator-max-microvolt = <3300000>;
			};

			vcc_lan: REG9 {
				regulator-name = "vcc_lan";
				regulator-min-microvolt = <3300000>;
				regulator-max-microvolt = <3300000>;
			};

			vdd_10: REG10 {
				regulator-name = "vdd_10";
				regulator-min-microvolt = <1000000>;
				regulator-max-microvolt = <1000000>;
				regulator-always-on;
			};

			vcc_18: REG11 {
				regulator-name = "vcc_18";
				regulator-min-microvolt = <1800000>;
				regulator-max-microvolt = <1800000>;
				regulator-always-on;
			};

			vcc18_lcd: REG12 {
				regulator-name = "vcc18_lcd";
				regulator-min-microvolt = <1800000>;
				regulator-max-microvolt = <1800000>;
				regulator-always-on;
			};
		};
	};
};

&i2c1 {
	status = "okay";
};

&i2c2 {
	status = "okay";
};

&i2c4 {
	status = "okay";
};

&i2c5 {
	status = "okay";
};

&i2s {
	#sound-dai-cells = <0>;
	status = "okay";
};

&spdif {
	status = "okay";
};

&io_domains {
	status = "okay";

	audio-supply = <&vcca_33>;
	flash0-supply = <&vcc_flash>;
	flash1-supply = <&vcc_lan>;
	gpio30-supply = <&vcc_io>;
	gpio1830-supply = <&vcc_io>;
	lcdc-supply = <&vcc_io>;
	sdcard-supply = <&vccio_sd>;
	wifi-supply = <&vcc_18>;
};

&pinctrl {
	pcfg_output_high: pcfg-output-high {
		output-high;
	};

	pcfg_output_low: pcfg-output-low {
		output-low;
	};

	pcfg_pull_up_drv_12ma: pcfg-pull-up-drv-12ma {
		bias-pull-up;
		drive-strength = <12>;
	};

	act8846 {
		pmic_int: pmic-int {
			rockchip,pins = <0 4 RK_FUNC_GPIO &pcfg_pull_up>;
		};

		pmic_sleep: pmic-sleep {
			rockchip,pins = <0 0 RK_FUNC_GPIO &pcfg_output_low>;
		};

		pmic_vsel: pmic-vsel {
			rockchip,pins = <7 1 RK_FUNC_GPIO &pcfg_output_low>;
		};
	};

	gmac {
		phy_int: phy-int {
			rockchip,pins = <0 9 RK_FUNC_GPIO &pcfg_pull_up>;
		};

		phy_pmeb: phy-pmeb {
			rockchip,pins = <0 8 RK_FUNC_GPIO &pcfg_pull_up>;
		};

		phy_rst: phy-rst {
			rockchip,pins = <4 8 RK_FUNC_GPIO &pcfg_output_high>;
		};
	};

	sdmmc {
		/*
		 * Default drive strength isn't enough to achieve even
		 * high-speed mode on firefly board so bump up to 12ma.
		 */
		sdmmc_bus4: sdmmc-bus4 {
			rockchip,pins = <6 16 RK_FUNC_1 &pcfg_pull_up_drv_12ma>,
					<6 17 RK_FUNC_1 &pcfg_pull_up_drv_12ma>,
					<6 18 RK_FUNC_1 &pcfg_pull_up_drv_12ma>,
					<6 19 RK_FUNC_1 &pcfg_pull_up_drv_12ma>;
		};

		sdmmc_clk: sdmmc-clk {
			rockchip,pins = <6 20 RK_FUNC_1 &pcfg_pull_none_12ma>;
		};

		sdmmc_cmd: sdmmc-cmd {
			rockchip,pins = <6 21 RK_FUNC_1 &pcfg_pull_up_drv_12ma>;
		};

		sdmmc_pwr: sdmmc-pwr {
			rockchip,pins = <7 11 RK_FUNC_GPIO &pcfg_pull_none>;
		};
	};

	usb_host {
		host_vbus_drv: host-vbus-drv {
			rockchip,pins = <0 14 RK_FUNC_GPIO &pcfg_pull_none>;
		};
	};
};

&saradc {
	vref-supply = <&vcc_18>;
	status = "okay";
};

&sdmmc {
	bus-width = <4>;
	cap-mmc-highspeed;
	cap-sd-highspeed;
	card-detect-delay = <200>;
	disable-wp;
	pinctrl-names = "default";
	pinctrl-0 = <&sdmmc_clk>, <&sdmmc_cmd>, <&sdmmc_cd>, <&sdmmc_bus4>;
	vmmc-supply = <&vcc_sd>;
	vqmmc-supply = <&vccio_sd>;
	status = "okay";
};

&tsadc {
	rockchip,hw-tshut-mode = <0>;
	rockchip,hw-tshut-polarity = <0>;
	status = "okay";
};

&uart2 {
	status = "okay";
};

&uart3 {
	status = "okay";
};

&usbphy {
	status = "okay";
};

&usb_host1 {
	status = "okay";
};

&usb_otg {
	/*
	 * The otg controller is the only system power source,
	 * so needs to always stay in device mode.
	 */
	dr_mode = "peripheral";
	status = "okay";
};

&vopb {
	status = "okay";
};

&vopb_mmu {
	status = "okay";
};

&vopl {
	status = "okay";
};

&vopl_mmu {
	status = "okay";
};

&vpu_mmu {
	status = "okay";
};

&vpu_service {
	status = "okay";
};

&hevc_mmu {
	status = "okay";
};

&hevc_service {
	status = "okay";
};

&wdt {
	status = "okay";
};

[tiandaozhang]

OK I try your suggestion.
I had try in this way:
sound {
compatible = "simple-audio-card";
simple-audio-card,format = "i2s";
simple-audio-card,name = "DW-HDMI";
simple-audio-card,mclk-fs = <512>;

            simple-audio-card,codec {
                    sound-dai = <&hdmi>;
            };

            simple-audio-card,cpu {
                    sound-dai = <&i2s>;
            };
    };

There isn't soud in the I2S port.

I think 'rockchip,tinker-codec' just is a driver's name . After I add below info in your file.
rk3288-miqi.zip

&hdmi {
#address-cells = <1>;
#size-cells = <0>;
#sound-dai-cells = <0>;

    ddc-i2c-bus = <&i2c5>;
    status = "okay";

};
I can find sound device in using 'aplay -l' command, else no sound card found.
volumio@Volumio:~$ aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: rockchiptinkerc [rockchip,tinker-codec], device 0: ff890000.i2s-i2s-hifi i2s-hifi-0 []
Subdevices: 1/1
Subdevice #0: subdevice #0

When I play music , no data output from I2S port.

[tiandaozhang]

You can also keep the spdif node, but I think it's only useful when you have an actual SPDIF output, like on Firefly devices.

I test I2S and SPDIF port in real environment with DAC codecs.
mini_board.zip

[Miouyouyou]

Nice setup !

Hmm, I'll try to understand what's going on in the driver tonight.

As a last try, could you try to replace the sound-dai = <&i2s>; by sound-dai = <&spdif>; And see how it fares ?

Also, could you provided the output of :

dmesg | grep -i I2S
dmesg | grep -i SPDIF

?

[tiandaozhang]

I test replaced dts file. The output log is below:
miqi_log_170126.txt

Expect your test result.....

[Miouyouyou]

So, I've checked both drivers and indeed the simple-audio driver just output nothing at all, while the rk3288-analog-hdmi just cannot work correctly using the setup provided in their documentation. I always get a rk3288-snd-hdmi-analog sound: ASoC: CODEC DAI hdmi-hifi.0 not registered when trying to use the rk3288-analog-hdmi driver with the ES8388 CODEC.

The setup outputting nothing with simple-audio was this one :

	sound {
		compatible = "simple-audio-card";
		simple-audio-card,format = "i2s";
		simple-audio-card,name = "rockchip,meow";
		simple-audio-card,mclk-fs = <512>;
		simple-audio-card,bitclock-master = <&dailink0_master>;
		simple-audio-card,frame-master = <&dailink0_master>;
		simple-audio-card,widgets =
			"Headphone", "Headphone Jack";
		simple-audio-card,routing =
			"Headphone Jack", "HP_OUT";
		dailink0_master: simple-audio-card,codec {
			sound-dai = <&hdmi>;
		};

		simple-audio-card,cpu {
			sound-dai = <&i2s>;
		};
	};

The thing is that the simple-audio driver is so generic that it clearly needs the right configuration to work correctly.

Unfortunately, I don't have a good setup for testing GPIO and I2S, nor do I have a scope for that, so this will have to wait a bit until I receive at least some cables and hardware to check how things perform... And some time to dwelve into the ALSA internals.

Currently, I have no idea what drives the I2S clocks and how to "play with the I2S" without creating yet another driver.
The simple-sound-driver have a few "widgets" that should be reflected on alsamixer but which are actually not reflected at all, for reasons I don't understand at the moment.
The rockchip-i2s is always probed correctly though.

So, until I find what's wrong, the only things I can suggest is to try playing with the simple-audio configuration and try hitting a few other places like :

• The ALSA mailing list Very high traffic with ALSA Audio professionals.
• The #linux-rockchip channel on irc.freenode.net (Low traffic with Rockchip engineers and upstream kernel submitters)
• The Rockchip section of the Armbian forum (Middle to low traffic with Rockchip boards users and a few kernel developers)
• The MQMaker's MiQi forum (You can ask benn there, the MiQi creator, if the I2S is wired up correctly, just in case).

If the same issue can also be reproduced on Linux 4.4 kernels provided by rockchip, you could also open an issue on Rockchip's kernel Github page.

Meanwhile, I'll provide the solution once I find it... This can take a while...

[tiandaozhang]

With changed miqi dts's file ,I tested MIQI kernel worked normal in I2S port and SPDIF port. The kernel link website is below: https://github.com/mqmaker/linux-rockchip/tree/miqi/release-4.4 , and the kernel version is 4.4. Thank you for your help, I will continue to explore this problem.

[ezequielgarcia]

@Miouyouyou @tiandaozhang This is a regression. I am working on a fix. Will keep you posted.

[ezequielgarcia]

@Miouyouyou @tiandaozhang Fixed here:
http://git.infradead.org/users/ezequielg/linux/commit/83ba31e1d31645ea7821ede72809061758193e66

I'll send this upstream soon.

[Miouyouyou]

Alright, I'll give this a test when I get home.

Thanks for you hard work 😃