It all started when I bought a new computer at home for my developer needs. Usually I fall to the other end of the gadget likeness spectrum. Specifically I avoid buying gadget stuff due to the unhealthy amount of time I found myself devoting to their configuration. However this time the conspiracy of things was masterfully executed to the point that I didn’t notice until it was too late (a blog and a github project later).
Actually If I want to be pedantic the first pieces has been fallen into place about 6 months before when my father switched ISP and he ended up with two new modems (ZTE-H108NS). Since I was the one that configured the new connections I preferred to use the existing modems in place. So I ended up with two brand new modems. At the time I thought “Hm? Maybe I should try to optimize my DSL line with a new modem and get rid of my trusty speedtouch 585“. I didn’t realize it then but that was probably the cornerstone that put in motion a series of events.
When finally my new computer arrived it had a feature that my previous 12 year computer was lacking. The mighty new feature was a working sleep/hibernate implementation. I immediately thought “Cool – I will be now able to hibernate my computer – It will not need to stay up all the time.”. Here is the train of thought:
- But what if I need something when I am away and my computer is turned off?
-
Well we should try this thing called WOL (Wake on LAN) – Another new feature for me.
Two weeks later after I have studied in detail all possible permutations of BIOS options, the fine differences between S4, power off and hybrid mode and wandered in forums I finally managed to master the WOL from my LAN.
- So what about the WAN? After all if I could use the LAN I could always switch on my computer by pressing the button – no need for WOL at all.
-
Wait! said an inner voice and the only advice I can give to myself from now on to not listen again that particular inner voice. If you are going to teach the modem about WOL maybe you should finally ditch the speedtouch 585. and invest some time to the new ZTE-H108NS modem said the voice.
DSL Line Optimization
So I switched modems – I configured the new modem and went to device status page in order to check on the DSL link speed. It was a gapping 12MBit/s exactly the same with the speedtouch 585. Given that my house is 35 meters away from the OTE (Greek ISP provider) building this is unacceptable. I picked up the phone and called my ISP. 15′ later my modem was syncing in 18MBit/s. Not bad.
Further research suggested the following command line (telnet interface of the modem)
#wan dmt2 set snrmoffset 4096 4096 #wan adsl reset
which happily increased my downlink to 22MBit/s. I thought let’s call it a day on the speed front and let’s concentrate back in the energy footprint front.
WOL over WAN
I was able to send the WOL over the internet with a minor arp tables hack. Specifically I had to map another address, let’s say 192.168.1.254 to the broadcast address.
#arp -s 192.168.1.254 FF:FF:FF:FF:FF:FF
Then on Advanced Setup/NAT/Virtual Server I setup a rule for ports 7-9 that goes to the internal ip address 192.168.1.254 which due to the above arp hack ends up as a broadcast packet waking up my computer. The following command wakes up my computer from anywhere in the world.
$wakeonlan -i `host myhost.mydomain.com | awk '{print $NF;}'` de:ad:ca:fe:be:ef
where myhost.mydomain.com is my Dynamic DNS hostname and de:ad:ca:fe:be:ef is the MAC address of my computer.
Dynamic DNS
The situation now begs the question: If the computer is turned off who is gonna maintain the Dynamic DNS entry up to date? Normally this is a job for the modem router because it has intimate knowledge of the external ip address and its changes. I am already a client of namecheap and I would like to use that provider for DDNS. Of course since the Internet community hasn’t succeed so far to standardize the DDNS updates protocol consequently the ZTE-H108NS modem doesn’t have support for namecheap.
Samba woes
In the meantime my wife had an unusual request. She wanted to print like it was the most natural thing of the world. I responded to the request lightly, full of self confidence. I installed samba and cloned the configuration from the backup of my previous setup. Yep that’s right. I am that good. I have a backup. What could possibly go wrong?
Well for starters my new machine was not visible in the network neighborhood. This took a good 2-3 hours to figure it out and it was a lesson to humility. It turns out that:
- The modem DHCP server sets itself as WINS server.
- Although nmbd is running the smb.conf in the modem does not enable WINS support. Bummer!
- There is no option in the GUI to enable the WINS server
- The modem confuses the workgroup name with the NETBIOS and vice versa.
The Decision
At this point insurmountable pressure has been gathered. I needed to hack the modem in order to fix the following itches ^H^H items.
- DSL Line setup
- Arp table setup for WAN WOL support
- Enable the WINS server in smb.conf
- namecheap Dynamic DNS support
These problems are not fixable without modifying the root filesystem or uploading custom firmware. Some of them require just a configuration file change but the namecheap Dynamic DNS support requires to modify and rebuild a MIPS executable.
Reverse Engineering -Know your enemy
Let’s try to find some info on this thing. I opened the case and I found these chips:
- Chassis PCB – BZRD0 V1.0
- CPU – U100 TrenChip TC3162U-LQ128G
- SDRAM- EtronTech EM6381165TS-6G
- SPI Flash – winbond 25Q16BVSIG
Let’s enter the modem via telnet:
$ telnet 192.168.1.1 Trying 192.168.1.1... Connected to 192.168.1.1. Escape character is '^]'. login: admin Password: # cat /proc/cpuinfo system type : TrendChip TC3162U SOC processor : 0 cpu model : R3000 V0.1 BogoMIPS : 330.95 wait instruction : no microsecond timers : no tlb_entries : 32 extra interrupt vector : no hardware watchpoint : no ASEs implemented : shadow register sets : 1 VCED exceptions : not available VCEI exceptions : not available unaligned accesses : 187256836 # # free total used free shared buffers Mem: 29196 27936 1260 0 2088 Swap: 0 0 0 Total: 29196 27936 1260 # ls -l / drwxr-xr-x 2 17431 10513 389 bin drwxr-xr-x 4 17431 10513 46 boaroot drwxr-xr-x 6 17431 10513 535 dev lrwxrwxrwx 1 17431 10513 8 etc -> /tmp/etc drwxr-xr-x 4 17431 10513 1507 lib lrwxrwxrwx 1 17431 10513 11 linuxrc -> bin/busybox dr-xr-xr-x 64 0 0 0 proc drwxr-xr-x 2 17431 10513 138 sbin drwxr-xr-x 7 0 0 0 tmp drwxr-xr-x 4 17431 10513 107 userfs drwxr-xr-x 7 17431 10513 67 usr lrwxrwxrwx 1 17431 10513 8 var -> /tmp/var # ls /etc/ AppFilter.sh ipv6_acl.sh nat_pvc9_0 RT30xxEEPROM.bin isp0.conf nat_pvc9_1 UrlFilter.sh isp1.conf nat_pvc9_2 Wireless isp11.conf nat_pvc9_3 acl.sh l7-protocols nat_pvc9_4 add_rule_for_dhcp.sh lanAlias0.conf nat_pvc9_5 adsl.conf lan_rip.conf nat_pvc9_6 adsl.sh lanconfig.sh nat_pvc9_7 autoexec.sh mac.conf ntp.sh bftpd.conf nat_pvc0 passwd config nat_pvc1 ppp ddns.conf nat_pvc10 protocols defaultWan.conf nat_pvc10_0 radvd.conf devInf.conf nat_pvc10_1 resolv.conf device_hostindex.conf nat_pvc10_2 resolv_ipv4.conf dhcp6c.conf nat_pvc10_3 resolv_ipv6.conf dhcp6s.conf nat_pvc10_4 ripd.conf dhcpd nat_pvc10_5 route6.sh dnsmasq.conf nat_pvc10_6 samba dproxy.conf nat_pvc10_7 services ethertypes nat_pvc2 shaper firewall.conf nat_pvc3 snmp fstab nat_pvc4 snmpd.conf.tmp fwTCver.conf nat_pvc5 sysconfig fwver.conf nat_pvc6 system group nat_pvc7 trx_config hosts nat_pvc8 udhcp_lease igd nat_pvc8_0 udhcpd.conf igmpproxy.conf nat_pvc8_1 udhcpd.fon.conf inetd.conf nat_pvc8_2 udhcpd.fon.leases init.d nat_pvc8_3 usb_modeswitch.conf inittab nat_pvc8_4 usb_modeswitch.d inittab_no_ra_menu nat_pvc8_5 usb_modeswitch_cfg.conf inittab_ra_menu nat_pvc8_6 usertty iproute2 nat_pvc8_7 xml ipupdate.conf nat_pvc9 zebra.conf # ls /userfs/bin/ auto_mount_dongle hello ripd bftpd igmpproxy skbmgr boa inetd smbd cfg_manager iwpriv smbpasswd chat mtd snmpd dhcp6c nmbd tcapi dhcp6s ntfs-3g tftpd dhcrelay ntpclient tr69 dnsmasq pppoe-relay usb_auto_mount ethphxcmd radvd vconfig ez-ipupdate restore_linos_info zebra # ls /usr/script/ AppFilterStop.sh ipv6_filter_forward_start.sh RebootScript ipv6_filter_forward_stop.sh UrlFilterStop.sh ipv6rd_start.sh acl_stop.sh lanAlias_start.sh before_tr069_download.sh lanAlias_stop.sh before_web_upgrade.sh nat_start.sh br_conf.sh nat_stop.sh ddns.sh ntpclient.sh ddns_run.sh ppp_start.sh dhcp6c_script restart_boa.sh dmz.sh samba.sh dongle_dial_off.sh samba_add_dir.sh dongle_dial_on.sh settime.sh ether_mac.sh spi_fw_start.sh filter_forward_start.sh spi_fw_stop.sh filter_forward_stop.sh udhcpc.sh fw_start.sh udhcpc_nodef.sh fw_stop.sh upgrade_firmware.sh getnow.sh urlfilter_start.sh ipaddr_mapping.sh urlfilter_stop.sh ipfilter.sh vserver.sh ipfilter_start.sh wan_start.sh ipfilter_stop.sh wan_start_ipv4.sh ipmacfilter_stop.sh wan_start_ipv6.sh ipv6_acl_stop.sh wan_stop.sh # mount /dev/mtdblock3 on / type squashfs (ro) proc on /proc type proc (rw) ramfs on /tmp type ramfs (rw) devpts on /dev/pts type devpts (rw) usbfs on /proc/bus/usb type usbfs (rw) # cat /proc/mtd dev: size erasesize name mtd0: 00010000 00010000 "bootloader" mtd1: 00010000 00010000 "romfile" mtd2: 000e9560 00010000 "kernel" mtd3: 004cf000 00010000 "rootfs" mtd4: 007a0000 00010000 "tclinux" mtd5: 00040000 00010000 "reservearea"
Some notes on what’s going on:
- The CPU is TrendChip 3162U. Looks like there are a lot of products that utilize this CPU and may use the TrendChip firmware.
- It has 32MB of memory.
- Looks like it has 16MB of flash.
- The rootfs is a standard mips uclibc based on busybox.
- The /etc directory is symlinked to /tmp which is a tmpfs.
- The /etc directory is created mostly by scripts living at /usr/script directory.
- The scripts are called by an executable named /userfs/bin/cfg_manager.
- The WEB GUI lives in /boaroot.
- The WEB GUI breaks out to cfg_manager via a socket interface. Not so sure about that.
- USB sticks are mounted under /tmp/mnt/usb1_1 and are automatically exported via samba. Question: Is it possible via a USB hub to mount multiple disks? That would be cool to actually convert a modem to a NAS.
- The flash is partitioned and looks like that some parts of it map directly to the tclinux.bin firmware image.
Reverse Engineering – binwalk
Let’s start with firmware 1.07 named tclinux.bin. Note the the name tclinux.bin is checked by the firmware during update and it is quite possible that the modem will refuse to update its firmware if the file doesn’t have that name.
#apt-get install binwalk $binwalk tclinux.bin DECIMAL HEXADECIMAL DESCRIPTION -------------------------------------------------------------------------------- 256 0x100 LZMA compressed data, properties: 0x5D, dictionary size: 8388608 bytes, uncompressed size: 3232020 bytes 955744 0xE9560 Squashfs filesystem, little endian, version 3.0, size: 5036741 bytes, 986 inodes, blocksize: 65536 bytes, created: Tue Feb 11 09:50:54 2014
So the first 256 bytes is the header and then something else and much later at 0xE9560 starts a squashfs. Let’s dig a bit more.
$binwalk -e tclinux.bin $ls _tclinux.bin.extracted/ 100 100.7z _100.extracted E9560 E9560.squashfs $cd _tclinux.bin.extracted $binwalk 100 DECIMAL HEXADECIMAL DESCRIPTION -------------------------------------------------------------------------------- 1430825 0x15D529 LZMA compressed data, properties: 0xD0, dictionary size: 131072 bytes, uncompressed size: 277750168 bytes 2646072 0x286038 Linux kernel version "2.6.22.15 (root@mbjsbbcf01) (gcc version 3.4.6) #1 Tue Feb 11 1c version 3.4.6) #1 Tue Feb 11 10:17:49 CST 2014"
So the firmware image looks like this
- 0-0x100 Header
- secondary bootloader (Couldn’t figure which was it)
- Linux kernel + initrd (Not sure which order)
- 0x09560 squashfs
- padding to 4K
The header has the following fields
- 0x00 magic number = 0x32524448
- 0x04 magic device (or header size – not sure) = 0x100
- 0x08 tclinux size = actual file size in bytes with header included
- 0x0C tclinux checsum = CRC – XOR checksum see below how to calculate
- 0x10 firmware version string = “7.0.1.0\n” in my case
- 0x30 a newline = “\n”
- 0x50 squashfs offset = 0x9560 in my case
- 0x54 squashfs size = size of squashfs size as reported by binwalk 5038080 bytes padded to 4096 (0x1000) sector 5038080 (0x4CE000)
- 0x5C model string = “3 6035 122 74\n” in my case
The program tcrevenge can be used to check an existing firmware file or calculate the checksum and generate a proper header for a new one.
Question: How to compare the live system with the firmware image?
Answer: We can put a usb stick to the modem (sometime requires reboot with the usb stick inside the jack) and use samba to copy files around.
for i in 0 1 2 3 4 5; do echo cat /dev/mtd$i > /tmp/mnt/usb1_1/p$i; done;
Notes:
- p0 (partition 0) is 64k has the primary bootloader. Running strings against it is instructive indeed. It includes a minimal web server and it should be (never done it) possible to recover the modem from a bad firmware image. That means it must have the same CRC algorithm with the normal software for uploading firmware. The contents of this partition do not change with a firmware update.
- p1 is the configuration file as saved by the WEB interface. For some reason it is called romfile.
- p2 is the secondary bootloader and probably the kernel and some initrd image. The contents of the partition are exact match with the firmware image tclinux.bin (header included) up to the squashfs image as extracted by binwalk above.
- p3 is the squashfs starting at 0x9560 of the tclinux.bin.
- p4 holds the tclinux.bin verbatim so the primary bootloader can recover the modem
- p5 is a reserved partition
Question: is it possible to modify the contents of the root (squashfs) filesystem?
Answer: We will need firmware-mod-kit in order to cope with the unusual LZMA compression and the older version number.
$git clone https://code.google.com/p/firmware-mod-kit/
Follow the instructions and build the firmware-mod-kit utilities. To uncompress and extract the filesystem you will need root user capabilities in order to create the device nodes and other weird files.
$sudo ../firmware-mod-kit/src/squashfs-3.0/unsquashfs-lzma E9560.squashfs created 731 files created 115 directories created 91 symlinks created 48 devices created 0 fifos $ls E9560.squashfs squashfs-root
After making modifications to the root filesystem. Here is how to pack it up again.
rm E9560.sq; ../firmware-mod-kit/src/squashfs-3.0/mksquashfs-lzma squashfs-root squashfs-root.sq
The file squashfs-root.sq contains the modified squashfs root image.
Now we need to extract the kernel so we can assemble back the modified tclinux.bin. Here is a one line to do it:
$dd if=tclinux.bin.orig of=kernel skip=256 count=`binwalk ../tclinux.bin.orig | awk '/Squash/ {print $1 - 256;}'` bs=1
The tclinux.bin.orig in the above command line is the original firmware image.
The tcrevenge program will output the header and the necessary padding and a nice command line suggestion to create the new tclinux.bin.
$./tcrevenge -k kernel -s squashfs-root.sq -o header -p padding Creating necessary squashfs paddingfile padding 1352 Magic number: 0x32524448 at 0x00 Magic device: 0x00000100 at 0x04 tclinux.bin size: 5993824 (0x005B7560) at 0x08 tclinux.bin checksum: 0xD4BEA4AA at 0x0C Firmware version at 0x10: 7.0.1.0 squashfs offset: 955488 (0x000E9460) at 0x50 squashfs size: 5038080 (0x004CE000) at 0x54 Model at 0x5C: 3 6035 122 74 Writing header to header. Create image with cat header kernel squashfs-root.sq padding > tclinux.bin $
Update 29/3/2015 15:30: The disassembly of the MIPS executables that led to the reverse engineering and the C reimplementation of the CRC algorithm in tcrevenge is the subject of another article.
Modifications
Now that we can build a new modified firmware at request let’s proceed with the actual modifications we would like to achieve.
DSL Line setup
That’s an easy one and it does not require firmware changes at all. The configuration file is called romfile.cfg. It is an XML file and has a section named Autoexec.
<Autoexec> <Entry cmd1="w dmt2 db tlb 2b" cmd2="wan ghs set multi_number 3 3" cmd3="wan dmt2 set largeD 2" cmd4="w dmt eoc dyingasp off" cmd5="w dmt2 set lpr off" cmd6="echo 1 > /proc/tc3162/port_reverse" cmd7="wan dmt2 set snrmoffset 4096 4096"/> </Autoexec>
Note the extra cmd7 added by me manually. This is enough to boost modem sync up to 22 MBit/s in my case. I didn’t find any script in /usr/script that generated /etc/autoexec.sh but the /userfs/bin/cfg_manager matches autoexec. So probably /etc/autoexec.sh is generated by the cfg_manager binary. Here is how /etc/autoexec.sh looks like:
# cat autoexec.sh w dmt2 db tlb 2b wan ghs set multi_number 3 3 wan dmt2 set largeD 2 w dmt eoc dyingasp off w dmt2 set lpr off echo 1 > /proc/tc3162/port_reverse wan dmt2 set snrmoffset 4096 4096
Note that you have to see that inside the modem via telnet since /etc/autoexec.sh is automatically generated on boot time and it is not available in the expanded squashfs partition.
Arp table setup for WAN WOL support
It took a bit of trial and error but finally I managed to guess the correct script to edit. It is not immediately obvious because scripts are being run by the mysterious cfg_manager binary. The right script is /usr/script/nat_start.sh and here is the patch.
diff -ur ../_tclinux.bin.extracted.orig/E9560/usr/script/nat_start.sh E9560/usr/script/nat_start.sh --- ../_tclinux.bin.extracted.orig/E9560/usr/script/nat_start.sh 2014-02-11 04:50:45.000000000 +0200 +++ E9560/usr/script/nat_start.sh 2015-02-27 20:37:59.908163151 +0200 @@ -62,3 +62,5 @@ # /usr/script/ipaddr_mapping.sh add $i $j # fi #done + +/sbin/arp -s 192.168.1.254 FF:FF:FF:FF:FF:FF
WINS support
This one was easier. The /usr/script/samba.sh is generating the /etc/samba/smb.conf. Here is the patch that enables WINS support.
diff -ur ../_tclinux.bin.extracted.orig/E9560/usr/script/samba.sh E9560/usr/script/samba.sh --- ../_tclinux.bin.extracted.orig/E9560/usr/script/samba.sh 2015-02-06 23:18:17.953887683 +0200 +++ E9560/usr/script/samba.sh 2015-02-06 22:49:31.920892757 +0200 @@ -17,6 +17,7 @@ netbios name = $NETBIOS_NAME server string = Samba Server workgroup = $WORKGROUP +wins support = yes security = user guest account = $GUEST log file = /var/log.samba
To be continued…
What is missing now is adding support for namecheap DDNS which belongs to another article.
I’m very glad to see that someone actually bothers to analyze and mod the rubbish Greek ISP’s provide keep up the good work.
LikeLike
Thanks for the kind words. It was a really fun project.
Stay tuned for the next episode: compiling minidlna for this modem / router 🙂
LikeLike
well i have managed to solve some of the mysteries regarding this router if you are interested to collaborate let me know.
LikeLike
Sorry for the late reply.
Sure I am interested if you have something available publicly to read that would be great.
If the information is not public / Free I am not interested.
LikeLike
i belive you noticed the cwmp upgrade!!! it came baring gifts……
LikeLike
Is this a firmware update? I have all the work so far with 1.0.7m cwmp is a remote management TR-069 thing.
Generally I don’t trust them and prefer them closed (by changing the password)
LikeLike
yea they deploy 1.1.7n with encrypted configuration, closed telnet some hardening but in the lack of JTAG to dump it ( its not available for download yet only from cwmp) i downgraded to 1.0.7 im developing something with telnet and i don’t have time to find another way in….
ps: Are you interested to collaborate for further analysis?….
LikeLike
sorry i didn’t notice the second reply the work i have done for the specific router is not yet public but the project im indenting to expand with functionality involving this router is already public https://github.com/ppanos/Helepolis
LikeLike
I too own this router and I’m interested in the nas/samba function but the thing is that they have limited the network transfer speed to everything mounted on the usb to 1.5mb/s max. I would need more. Is there an easy way to work it out?
Thanks for all.
LikeLike
Hmm… Is this 1.5Mbytes/sec or 1.5Mbits/sec? If it is MBits it is too low if it is MBytes then it is equivalent with 12Mbits/sec which is the full speed of USB 1. I doubt though that they would do such a thing – I mean who manufactures usb1 ports and controllers any more?
If I were you I would concentrate in the following possibilities:
1) What kind of disk do you have connected? Is it usb2? What the dmesg says when you are connecting. Is it a usb stick? Some sticks have terrible performance. Does the kernel sees it as usb2 device?
2) Does the mips cpu has enough power to sustain peak performance during transfer? Login to the modem and monitor the output of top.
and finally one question: What kind of filesystem did you mount for your tests?
Vassilis
LikeLike
It is 1.5megabytes/s max (usally around 1)
1. The disk connected is a 1TB ntfs which is capable of around 15-20mb/s via my usb 2 port
2. I aggree that cpu probably does not have enouph power and that’s the reason they have limited the speed .. (https://www.ote.gr/en/web/guest/help-and-support/internet/-/support/article/1034896)
Thanks anyway
LikeLike
Hi. Interesing article. Do you know where I can download the 1.07 or 1.1.7n version of the firmware? It is removed from https://www.ote.gr/web/guest/help-and-support/internet/-/support/article/1034896.
Thank you for writing.
LikeLike
Hey! after the latest firmware, romfile.cfg is not editable, contains characters such as:
O뚨ùŸðP•þìÒzwØZxã“›ÐÝÒlÉ4¢#íÈ#gÌTIðå
LikeLiked by 1 person
@Vas,
That’s unfortunate…
Please share some details of your setup such as firmware version.
LikeLike
I just love you man. Keep it up!
LikeLike
really nice and helpful guide. Did you manage to install openwrt on that device?
LikeLike
Hi John,
No, I didn’t install openwrt.
I just used tcrevenge to modify the existing firmware. Porting openwrt to this modem is a much more ambitious project although the specifications of the modem have room for openwrt (flash 16MB, RAM 32MB).
I am sure that it is possible to use tcrevenge to create a flashable openwrt image but you first need to port the kernel in this board.
Existing firmware has a linux kernel but no sources. In theory they should provide the sources once you ask. Next step is to build openwt with these custom kernel patches.
Good luck if you try it.
However if manage to do it let me know to dedicate a section for you in the hall of fame https://vasvir.wordpress.com/2017/01/28/trendchip-firmware-zte-h108ns-derived-works/
LikeLike
I encountered a similar firmware, and the calculated CRC32 is different from the CRC32 given by the firmware. Can you help me see what’s wrong. My email: 411497976@qq.com
LikeLike
Hi,
Sorry can’t help you much. I would start with a very careful readout of tcrevenge output.
There are some command line options and one or two magic constants in the code that you can tweak.
I would run tcrevenge with a) the original firmware b) the constructed firmware and see where they differ…
LikeLike
I’m glad to contact you and get your reply. I also saw your article from a blog. It took some trouble to contact you.
I’ve been studying this firmware recently. I’ll sort it out when I’m free these days. I’ll ask you later.
My Linux foundation is very poor. If I do wrong, please don’t blame me.
LikeLike