Products > Technical Papers

Introduction to H.264 Video Compression Standard
Frank Wang  |  Software Engineer







ABSTRACT — In the IP camera industry, H.264 is the most popular video compression standard that provides the format for recording digital video. The H.264 standard was first published in 2003, with several revisions and updates since then. It has achieved a significant improvement in rate distortion efficiency relative to existing standards. It has made significant progress in compression efficiency and uses less capacity when data is stored or transmitted.



I – Introduction

In a typical application of H.264 such as video surveillance, video from a camera is encoded using H.264 to produce an H.264 bitstream. It is sent across a network to a decoder which reconstructs a version of the source video.

This standard exploits both temporal correlation and spatial correlation to remove the pixel redundancy in a video sequence. Furthermore, the high correlation between each syntax element is also used to predict and then represent the target syntax element. Thus, the interdependence between each syntax element is quite huge.



II – The H.264 Bitstream Structure

A coded picture consists of one or more slices that are made up of a series of coded macroblocks. An overview of the H.264 bitstream structure is in the following section.


A. Group of Pictures

A group of pictures (GOP) comprises a successive set of pictures which starts with a key picture [1]. In a closed-GOP coding structure, the key picture may be an instantaneous decoding refresh (IDR) picture that consists of one or more IDR slices or a special type of intra-coded slice. If an IDR picture is received at the decoder, all stored pictures that are in the decoded picture buffer (DPB) are marked “unused” immediately. Therefore, pictures preceding the IDR picture in coding order are no longer available for prediction.

In a hierarchical-P coding structure (i.e., coding order is equal to output order), P pictures are predicted from the pictures that are already encoded. This structure is designed for scenarios which require low-delay operation. A hierarchical-P coding structure example is presented in Fig.1.






B. Network Abstraction Layer Unit

An H.264 coded video sequence consists of a series of network abstraction layer units (NALUs). Each of them may include parameter sets, supplemental information, an entire coded picture, or parts of an coded picture [2]. A VCL NALU consists of a one-byte NALU header that is used to define the information within the NALU payload. There are three fields in the NALU header:

1. a 1-bit forbidden_zero_bit (F): indicates the data loss which can be used to trigger error concealment at the decoder 2. a 2-bit nal_ref_idc (NRI): signal the importance of NALU

3. a 5-bit nal_unit_type (NUT): signals the type of encapsulated byte sequence packets (EBSP) in the NALU

The most common NALUs are listed in Table I. The decoder can detect the NALU type in the NALU header to perform the different processes.






C. Macroblock Layer

The composition of a video coding layer (VCL) NALU is presented in Fig. 2. The macroblock is the basic unit for the coding process in a picture. The region size is 16×16 pixels and contains one 16×16 luminance sample and two 8×8 chrominance samples in 4:2:0 YUV format.



Figure 2: The composition of a VCL NALU, only one slice in a picture.





The macroblock information is presented by a lot of syntax elements that form the string of data bits (SODB). Because a SODB may or may not be byte-aligned, it is necessary to transfer the SODB into the byte-aligned raw byte sequence payload (RBSP) for the specification requirements. In order to prevent zero_byte (i.e., 0x00) and start_code_prefix_one_3bytes (i.e., 0x000001) from occurring in the NALU payload, an emulation prevention (i.e., 0x03) is inserted into the bitstream when 0x0000 is happening.

Figure 3 illustrates a general H.264/AVC encoder architecture; the related encoding process is described as follows: a raw picture is divided into many non-overlapping macroblocks, and then a prediction macroblock is generated by intra or inter prediction and subtracted from the target macroblock to form a residual macroblock. Then, the residual macroblock is transformed and quantized according to the selected quantization parameter (QP). Finally, the residual macroblock is coded into the bitstream by entropy coding. There are two entropy coding modes allowed in H.264/AVC; one is context-based adaptive variable length coding (CAVLC), and the other is context-based adaptive binary arithmetic coding (CABAC). Only the CAVLC entropy coding tool is supported in the baseline profile.



Figure 3: The H.264/AVC encoder architecture





1. Intra Prediction

An intra macroblock (I-macroblock) is coded using spatial correlation. The slice that has the same slice group number can act as the predicted reference for intra prediction in the I picture.

Figure 4 exhibits a set of intra 4×4 prediction modes that are available in H.264, including a DC and 8 angular predictions to make up different combinations of prediction. These prediction modes are suitable for the complex region.



Figure 4: Intra 4×4 prediction modes.






Figure 5: Intra 16×16 prediction modes.






Figure 5 exhibits a set of 4 intra 16×16 prediction modes. The pixels that are highlighted in red line are used to form the prediction for the current block. A bigger prediction block size tends to give less precision, but fewer bits are required to code the prediction mode.

Figure 6 shows an example for intra 4×4 macroblock from the Elecard StreamEye software. The decoded macroblock is the combination of the predicted macroblock and the residual macroblock. The highlighted red block is coded by the intra 4×4 vertical prediction mode because the original texture has the vertical texture; the encoder attempts to select the best mode during rate distortion optimized (RDO) mode selection. The best coding mode of a macroblock is based on the trade-off between the bitrate and distortion cost that is controlled by the QP. A smaller QP tends to select the mode that gives the least distortion, allowing a higher bitrate.



Figure 6: An example for intra 4×4 macroblock. (a) Predicted macroblock. (b) Residual macroblock. (c) Reconstructed macroblock.





2. Inter Prediction

An inter macroblock (P-macroblock) is coded using the temporal correlation. The prediction block is formed from previously coded pictures stored in the DPB. There are many partition sizes for selection in H.264/AVC. The ultimate goal is to find a trade-off between the total bits that are used for coding the motion vector and the residual data and the distortion cost. As can be seen in Figure 7, the offset between the co-located block and the best match block (i.e., the car tire) in the reference frame is the so-called motion vector [3].



Figure 7: Finding a suitable prediction block.






3. Residual Data

The output of the prediction process is a residual block, created by subtracting the prediction block from the current block, and a set of parameters signaling the intra prediction type or describing how the motion block was estimated. Figure 8 depicts the residual blocks in an intra 4×4 macroblock for YUV 4:2:0 sampling; the number labeled in the block is the transmission order.



Figure 8: The residual blocks in an intra 4×4 macroblock, 4:2:0 sampling.






Figure 9: The residual blocks in an intra 16×16 macroblock, 4:2:0 sampling.






The residual data have the most important influence on the bitrate of a bitstream. One way of controlling bitrate is simply to try and enforce a constant number of bits per coded frame, by measuring the output bitrate and feeding it back to control QP. Increasing QP reduces coded bitrate and decreasing QP increases coded bitrate.

D.Profiles

The H.264 standard specifies many syntax and decoding algorithms, which cover a wide range of potential video scenarios. A profile places limits on the algorithmic capabilities required of an H.264 decoder. Each Profile is intended to be useful to a class of applications. Hence, a decoder conforming to the main profile of H.264 only needs to support the tools contained within the main profile. For example, the baseline profile may be useful for low-delay or conversational applications such as video conferencing, with relatively low computational requirements. The main profile may be suitable for basic television applications such as standard definition TV services. The high profiles add tools to the main profile which can improve compression efficiency especially for higher spatial resolution services such as high definition TV.

Table II lists the coding tools supported by the baseline, extended, main and high profiles. We can choose a suitable profile for the special scenario.



Table II





III – Analysis Of Practical Stream

As can be seen in Figure 10, we capture a practical stream from our IP camera and analyze the stream by the Elecard’s StreamEye analyzer software. In this case, a main profile sequence that starts with an IDR slice (red bar) followed by P slices (blue bar). It is obvious that the bitrate of the IDR picture is greater than the bitrate of the P picture. Besides, the entropy coding method is CABAC and the frame rate is 30.






IV – Conclusion

In this paper, we offered an overview of the H.264 video compression standard. It has significantly improved in rate-distortion efficiency relative to existing standards. It describes the significant progress on compression efficiency that uses less capacity when it is stored or transmitted. There are two attractive video compression standards, H.265 and AV1. However, there is still a long way to go to meets the needs of manufacturer and customer.



References

1. M. Wien, High Efficiency Video Coding: Coding Tools and Specification, Berlin: Springer-Verlag, 2015.
2. I. E. Richardson, The H.264 Advanced Video Compression Standard, 2nd ed., New York: Wiley, 2010.
3. Y. Wang, J. Ostermann, and Y. Q. Zhang, Video Processing and Communications, Prentice Hall, 2001.



Download PDF >





See all Technical Papers >


RTSP Live Streaming
Frank Wang  |  Software Engineer







ABSTRACT — With the rapid evolution of the IP camera industry, almost every IP surveillance camera supports Real Time Streaming Protocol (RTSP) video streaming, which means the user can operate a media player to watch live views from anywhere. RTSP provides a way for users to control video and audio. Real-time Transport Protocol (RTSP) does not actually provide for the transfer of video signals and audio signals; it controls how packages are delivered.



I – Introduction

RTSP is a network control protocol that is designed for watching a live feed and controlling media sessions between end points. The transmission of streaming data itself is not a task of RTSP. Instead, RTSP uses a combination of reliable transmission over TCP (used for control) and best-efforts delivery over UDP (used for content) to stream content to users. Most RTSP servers use the Real-time Transport Protocol (RTP) in conjunction with Real-time Control Protocol (RTCP) for media stream delivery.



II – Analysis Of Network Packet

Let’s consider an interaction where the client and server will use a combination of TCP-based RTSP and UDP-based RTP and RTCP to deliver a video stream.


We capture the network traffic between our IP camera and clientside by Wireshark. The parameters of experimental scenario are shown in the table below.

A. RTSP

RTSP is a protocol used for transferring real-time multimedia data (e.g., audio and video) between a server and a client. It is a streaming protocol; this means that RTSP attempts to facilitate scenarios in which the multimedia data is being simultaneously transferred and rendered (i.e., video is displayed, and audio is played).

The server needs to maintain a session state to be able to correlate RTSP requests with a stream. The state transitions are depicted in Fig.1.



Figure 1: State machine transition diagram of RTSP server.






The basic RTSP requests are described as follows:

1. Options

The client will establish a TCP connection to port 554 on the server. An OPTIONS request returns the request types that the server will accept.






2. Describe

A DESCRIBE request includes an RTSP URL and the type of reply data that can be handled. This response includes the presentation description, typically in Session Description Protocol (SDP) format.






3. Setup

SETUP request specifies how a single media stream must be transported. The request contains the media stream URL and a transport specifier.

transport specifier. This specifier typically includes a local port for receiving RTP data (audio or video), and another for RTCP data (meta information). The server reply usually confirms the chosen parameters, and fills in the missing parts, such as the server’s chosen ports. track1 (audio):






– Client RTP port is 59842, client RTCP port is 59843
– Server RTP port is 6970, server RTCP port is 6971

track2 (video):






– Client RTP port is 59844, client RTCP port is 59845
– Server RTP port is 6972, server RTCP port is 6973

4. Play

A PLAY request will cause one or all media streams to be played.






5. Teardown

A TEARDOWN request is used to terminate the session. It stops all media streams and frees all session-related data on the server.






B. RTP

RTP is designed to carry the encoded audio and video data. RTP adds a header to each packet, which is then passed to the UDP for further processing.

RTP also provides a time-stamping function that allows multiple streams from the same source to be synchronized. Each form of payload (i.e., video and audio) has a specific way of being mapped into RTP.

Each source inserts time stamps into outgoing packet headers which can be processed by the receiver to recover the stream’s clock signal that is needed to correctly play video and audio clips.

As can be seen in Fig. 2, an entire frame can be identified as a sequence of packets ending with a packet having the RTP marker bit set.



Figure 2: Video and audio RTP packets.





For voice packets, the marker bit indicates the beginning of a talkspurt. Beginnings of talkspurts are good opportunities to adjust the playout delay at the receiver to compensate for differences between the sender and receiver clock rates as well as changes in the network delay jitter.

C. RTCP

RTCP is a bidirectional UDP-based mechanism to allow the client to communicate stream-quality information back to the server. This connection always uses the next incremental UDP port of the RTP source port. Fig. 3 shows how the three protocols work together in our experimental scenario.



Figure 3: The three main application protocols used in real-time streaming.





III – Conclusion

The RTSP provides a means for the user to control video and audio. RTSP does not actually provide for the transport of video signals and audio signals; instead, it allows these signals to be controlled by the user. Like a dispatcher for a delivery service, RTSP does not actually deliver packages; it controls when and how packages are delivered by other protocols, such as RTP.



References

1. M. Syme and P. Goldie, Optimizing Network Performance with Content Switching: Server, Firewall and Cache Load Balancing: Server, Firewall, and Cache Load Balancing, Prentice Hall, 2003.



Download PDF >





See all Technical Papers >


Reliable Packet Transmission with UDP
Yuan-Ping Chang  |  Software Engineer







I – Introduction

UDP (User Datagram Protocol) is an unconnected transport layer protocol in the OSI (Open System Interconnection) reference model, providing a transaction-oriented simple unreliable information transfer service. It is called “unreliable” because there is no handshake or verification of the data transfer.

It is not a connection type protocol, so it has the advantages of low resource consumption and fast processing speed. Therefore, audio, video, and normal data usually use UDP when transmitting, because losing even one or two data packets occasionally is not that big of a deal. We barely notice if a packet drops from a video stream or call.

UDP does not have a TCP handshake, acknowledgment, window, retransmission, congestion control, etc. UDP is a stateless transport protocol, so it is very fast when passing data. Without the mechanisms of TCP, UDP is less vulnerable to exploits than TCP.



II – Background

Due to the unreliability of UDP, it has been limited in some applications. Therefore, we have proposed a reliable UDP transmission mechanism to solve this problem. It can also be applied to other applications besides high-speed data transmission, such as Point-to-point technology (P2P), firewall penetration, multimedia data transmission, and so on.


III – How Reliable Packet Transmission With UDP Works

The sending and receiving parties are divided into a connection initiator and a receiver, and data transmission is performed through Reliable Packet Transmission with UDP.

The connection initiator first sends a UDP packet with a packet type of HELLO to the receiving end, and the source sequence number is a non-repeating sequence number generated by the connection initiator, which is only applicable to the conversation. The packet sequence number is 0.

When receiving the UDP packet whose packet type is HELLO, the receiving end replies with a UDP packet whose packet type is HELLO ACK. The source sequence number is the sequence number sent by the connection initiator, and the packet sequence number is 0.

After receiving the UDP packet with the packet type HELLO ACK, the connection initiator sends a reply to the UDP packet of the same content, and the connection is established. The connection initiator can start sending the UDP packet with the packet type DATA.

At the end of the connection, either end can send a UDP packet with a packet type of BYE. The source sequence number is the sequence number of the session phase, and the packet sequence number is 0. The other party will reply to the same content UDP packet after receiving it, and both parties can end the dialogue phase. This is illustrated in Fig.1.



IV – Data Transmission Phase

Both the originating end and the receiving end can transmit the data packet to the other party. The UDP packet of the packet type is DATA, the source serial number is the sequence number of the dialogue phase, the packet length is the length of the data to be transmitted, and the packet sequence number starts from 0—incrementally in order. After receiving the data packet transmitted by the other party, it must reply to the UDP packet whose packet type is DATA ACK. The source sequence number is the sequence number of the session phase, and the packet sequence number is the packet sequence number of the data packet received at the time.



Figure 1






ERROR Detection and Correct

In the data transmission phase, after the data packet is sent out, if the data packet is not received, the DATA ACK packet for the data packet must be resent to avoid the loss of the data packet during the transmission. The data receiving end shall establish a queue. After receiving the UDP packet of the packet type DATA, the data is temporarily stored in the queue according to the packet serial number. If the serial number is found to be misaligned, the data may be adjusted and then read sequentially. This is how to avoid data errors caused by different data packet arrival orders due to network delay.

Send Window

In the data transmission phase, the data transmitter defines the Send Window Size, and sends multiple data packets in sequence. After receiving the DATA ACK packet from the receiving end, it continues to transmit multiple data packets within the Send Window Size range instead of transmitting a single data packet and waiting for a DATA ACK to be sent before sending the next data packet. When the network is delayed, the Send Window can greatly reduce the time wasted by the data sender waiting for the DATA ACK packet. This is illustrated in Fig.2.

Slow ACK

In the data transmission phase, the data receiving end does not return the DATA ACK packet for each data packet, but after a fixed time, the DATA ACK packet is sent back and forth to the last data packet received, and the Send Window can reduce the transmission of a large amount of data.



Figure 2






The DATA ACK packet traffic caused by the time. This is illustrated in Fig 3.



Figure 3






Slow Start

The data sender’s Send Window Size will be adjusted to the appropriate size according to the network conditions. At the beginning of the data transfer, the data sender’s Send Window Size will increase from the defined minimum value to the maximum value over time, but the Send Window Size will be moderately reduced when the data transfer party detects that a packet loss occurs. To avoid the network congestion, the data transmitter continues to send a large number of data packets to make the network situation worse.



V – Conclusion

Different technologies can be selected according to different transmission scenarios. It is possible to choose a loose congestion mode or a specific retransmission mode, but no matter how you choose, it is based on Expense, Latency, and Quality. The trade-offs between the three can help developers find a better solution by combining the scenarios and balancing the triangular balance.



Download PDF >





See all Technical Papers >


High Speed and Reliable Protocol Based on UDP
Yuan-Ping Chang  |  Section Manager







ABSTRACT — UDP is short for User Datagram Protocol, providing a simple and efficient network transport protocol. UDP uses a short protocol mechanism to transmit packets, and does not guarantee whether packets are lost, sequenced, delayed, etc. during transmission. However, due to its simplicity and high efficiency, it is often used in applications such as sound and video transmission, which can tolerate packet loss and timeliness greater than reliability.



I – Introduction

In the field of P2P communication, due to the widespread application of Network Address Translation (NAT), network devices located in different NATs cannot directly connect with each other. Therefore, a series of NAT traversal technologies have been developed.

Among them, the most reliable and widely used P2P traversal NAT method is to use UDP packets for traversal. Since most NAT firewalls are more lenient for UDP packet transmission management, the connectivity of NAT traversal using UDP packets is much greater than other methods.

After the successful NAT traversal, the packets of various types of transport protocols are packaged as UDP packets for transmission, increasing the transmission reliability of the UDP packets.



II – Background

Due to the increasing popularity of high-definition surveillance equipment requiring large bandwidth, NAT traversal technology is indispensable when the equipment is under NAT.

Therefore, we need to have a reliable transmission protocol so that packets are sent in UDP, and the speed must meet the needs of high-quality image transmission.

When using UDP to transmit packets, high speed and reliable protocol based on UDP maintains the high-speed transmission rate of UDP and ensures the correctness and reliability of the packet contents during transmission.



Figure 1






The forty-first to fifty-sixth bits are the packet sequence numbers, and each packet has its own serial number. When the packet is delayed or lost, it can be reordered or re-sent to ensure the reliability of the data.


III – How High Speed and Reliable Protocol Base on UDP Works

In addition to the original UDP header, there is a self-defined data structure to achieve high-speed transmission and ensure the reliability of data. This is illustrated in Fig.1.

The first sixteen bits are the source sequence number, which allows both parties to discern whether the stream is from the same session. The seventeenth to twenty-fourth bits are packet types. This defines ten types of packets, allowing both parties to identify and process different types of packets. This is illustrated in Fig.2.

The twenty-fifth to the fortieth bits are the packet length, and the length information of the packet of both parties is provided for confirmation.


IV – Conclusion

When using UDP to transmit data, this communication protocol can ensure the transmission speed and reliability of the data. In the actual application of the product, the transmission bandwidth can reach 80Mbps. Even when the network is severely blocked and delayed, the transmission bandwidth can reach speeds above 30Mbps.



Figure 2




Download PDF >





See all Technical Papers >


BLE Direct Test Mode Through 2-wire UART Interface
Kerr Lu  |  Software Engineer












ABSTRACT — The Bluetooth standard defines Direct Test Mode (DTM) for RF PHY testing of Bluetooth low energy device. This standardization verifies that a basic level of system performance is guaranteed for all BLE products. We will introduce the method of perform the tests through a 2-wire UART interface.



I – Introduction

The RF validation of Bluetooth device uses a protocol called Direct Test Mode. It is described in the Bluetooth Core Specification versions 4.x and 5.0, Volume 6, Part F.

The purpose is to test the radio at the physical layer for things such as transmitting power and receiver sensitivity, which is useful for regulatory EMC testing.

The Direct Test Mode (DTM) is used to control the Device Under Test (DUT) and provides a report back to the TESTER. It has two alternate methods:



Figure 1: bluetooth protocol stack


1. Over HCI





2. Through a 2-wire UART Interface





In the next section, we describe the second way to perform the tests through a 2-wire UART interface.



II – Test Sequences

The DTM protocol enables communication between the DUT (Device Under Test) and the Tester. The test equipment that we usually use in the development phase is Anritsu MT8852B, which combines the upper and lower tester functions.




The Upper Tester has direct access to the DUT through a dedicated 2-wire connection which can enter commands to start and stop the RF test. The Lower Tester is a piece of lab equipment that will measure the RF activity and performance.



Figure 2-1: message sequence charts of transmitter test



Figure 2-2: message sequence charts of receiver test



III – Commands And Events

Command and Event behavior of 2-wire UART interface.




˙– Commands




1. CMD(command)




2. Frequency




3. Length




4. PKT(Packet Type)




Note: Vendor specific will be detailed in section

Events
– LE_Test_Status_Event




1. EV (event)




2. ST (status)




3. DC (don’t care)
– LE_Packet_Report_Event




1. EV(event)




2. ST(status)




IV – Vendor Specific

The standard 2-wire UART interface Command reserves binary value “11” at PKT field for Vendor Specific packet payload.




For instance, Nordic nRF52832 have four vendor options as below:




CARRIER_TEST

If column 3 (the length) of the packet is set to 0, an unmodulated carrier 2.

ST (the status) is turned on at the channel indicated by column 2 (the frequency).


CARRIER_TEST_STUDIO

If column 3 (the length) of the packet is set to 1, an unmodulated carrier is turned on at the channel indicated by nRFgo studio, which is an application for configuring Nordic chips and which also supports a range of radio testing.


SET_TX_POWER

If column 3 (the length) of the packet is set to 2, column 2 (the frequency) sets the TX power in dBm.

Valid settings on Nordic nRF52832 are -40, -20, -16, -12, -8, -4, 0, +3, +4dB. They can only be modified while no transmitter or receiver test is running.


Only the 6 least significant bits of Tx power value will be fit in. Take -4dBm as an example: The representation of 8-bit binary in decimal -4 is 11111100. Determine that the 6 least significant bits are 111100, then fit them into the frequency field of the packet payload.







The command of setting Tx power to -4dBm in hexadecimal form will be 0xBC0B.

SELECT_TIMER

If column 3 (length) is set to 3, column 2 (frequency) selects the timer for transmitter test timing. The valid values of the timer are 0, 1, 2.

Summarize the Vendor Specific packet payload into one format.






V – Debugging

All commands and events are binary messages, we need a simulator for monitoring the communication between two devices. Docklight is an application that can be simulated as a RS232/UART device and can manually create send/receive sequences from the communication data.

Using Nordic nRF52832 as a DUT, set Docklight to send the SET_TX_POWER of vendor commands, and check if the transmit power is correct through the RF power meter.



Set commands of sending sequence on Docklight:

1. DTM_tx set as LE_Transmitter_Test command (0x8000)

2. DTM_end set as LE_Test_Eed command (0xC000)

3. DTM_reset set as LE_Reset command (0x0000)

4. vender_value set as SET_TX_POWER of vendor commands

Click the button for vendor_value to set the Tx power, then click the DTM_tx button to start sending the sequence. DUT will response [RX]0x0000 every time an event is successful.



Figure 5-1: screenshot of Docklight simulating as a Tester



Figure 5-2: screenshot of power meter NI USB-5681 Soft Front Panel



References

1. Bluetooth Specification
Version 4.0 Volume 6 Part F, Bluetooth Special Interest Group.

2. Nordic Semiconductor Infocenter
https://infocenter.nordicsemi.com/index.jsp?topic=%2Fcom.nordic.infocenter.sdk5.v12.3.0%2Fble_sdk_app_dtm_serial.html

3. Nordic Semiconductor nRF5 SDK v12.3.0
Directory:nRF5_SDK_12.3.0_d7731ad/examples/dtm/direct_test_mode



Download PDF >





See all Technical Papers >


Software Power Table for QCA Dakota Platform
Griffin Wang  |  Software Engineer







ABSTRACT — Wireless products tend to be sold in different countries with different regulatory domains and different power tables. The original design in the QCA Dakota platform supports only 3 power tables: FCC, ETSI and MKK. In this document, we propose a flexible software solution to support more power tables for all countries.



I – Introduction

Wireless products have become increasingly popular in recent years making Wi-Fi connection available everywhere. But the transmit power limitation is different from country to country because the regulations in each region is different. The original design of QCA’s Dakota platform supports only 3 power tables: FCC, ETSI and MKK. But wireless products are sold to far more than countries than those that use these power tables. Countries require that we follow the local transmit power rule to limit the transmit power of wireless products. In our current situation, we will need to support India, Korea, Australia, New Zealand, and Canada. But the current architecture doesn’t have enough space to add more power tables. It’s a fixed format called the board data file, which is included as a binary file in firmware. QCA’s Wi-Fi driver relies on the country code to read the power table in the binary file.



II – Problem Statement

One solution is to separate the firmware image for different countries. Each firmware includes different board data files for the corresponding countries. But it will be hard to maintain the firmware image if it is modified by a firmware upgrade. Another solution is to include all countries’ board data files in one firmware image and change the board data file properly when selecting countries. In this solution, many board data files have to be included in one firmware version and need to replace the original one when changing countries. It not only increases the firmware size but needs to do some additional steps to replace the board data with the correct one and send to the target firmware.

Software Power Table for QCA Dakota Platform





III – Solution

In order to solve the above-mentioned problems, the solution provides a more flexible way to support multiple countries’ power tables. And it’s not necessary to add many board data files in the firmware, then replace the original one at run time. We follow the power table’s structure to add the power limitation for each country. Then it will be chosen when the corresponding country code is selected. The selected power table will be sent to the target firmware directly and applied to the radio. Its transmit power will then follow the settings in the power table for each channel.






IV – Structures Of Ctl Power Table And Country List

struct ctl_power_channel_pair {
u_int8_t ctl_channel;
u_int8_t ctl_power;
u_int8_t ctl_flag;
};
struct ctl_country_2G {
struct ctl_power_channel_pair country[];
};
struct ctl_country_5G {
struct ctl_power_channel_pair country[];
};
struct ctl_country_2G_Country1_2g[]={

/*11b*/ {}
/*11g*/ {}
/*11n ht20*/ {}
/*11n ht40*/ {}
}
……
/*11a*/ {}
/*11n ht20*/ {}
/*11n ht40*/ {}
/*11ac ht80*/ {}
}
struct ctl_country_5G_Country2_5g[]={}
……



V – Descriptions

1. Get a list of all countries that should be supported by the wireless products.

2. Follow the power table’s structure to create the power limitation for both 2.4G and 5G in different band and channels. Repeat step 2 for all of the countries in the country list.

3. Add a statement to decide the power table by the selected country code.

4. When the driver is loaded, it will read the original power table in board data file. When the user selects the country code, it will get the corresponding power table from the structure we created. Then overwrite the original power table.



VI – Conclusion

To include all countries’ power table in one software would be more flexible to support multiple countries, and we don’t need to maintain different firmwares or different boarddata files for each country or region. Besides, once we need to support more countries, we just need to add one more power table. Otherwise, if the power limitations are changed by the regulatory rule, we can modify the value in our power table with the new power table. But the limitation of this solution is it’s platform depended. It has to be changed when the platform and structure are changed. For example, 11ax would be different because the format of power table is changed.



Download PDF >





See all Technical Papers >


Bluetooth Low-Energy 5.0 Throughput Testing
Louis Wang  |  Senior Software Engineer












ABSTRACT — Bluetooth is a wireless technology used in many applications with newer versions coming out every year to meet the needs of users. The newest version contains the best improvements to date. The improvements mainly relate to the high speed LE 2M physical layer data rate and LE long-range transmission. This document will focus on throughput testing and analyze different parameter settings that cause different throughput results.


I – Introduction

We use 2 Nordic nRF52840-DK boards to do throughput testing. One of the boards acts as a central (master) and another acts as a peripheral (slave). First, we give an overview of the BLE connection procedure. Second, we talk about the GATT (Generic Attribute Profile) protocol factors, ATT_MTU, data length extension and connection interval. Making experiment to see how these factors affect the throughput. Finally, based on these experiments, we show how to optimize for maximum data throughput in BLE 5.



II – BLE Stack and Frequency Channels

SoftDevice S140 is a feature complete Bluetooth 5 qualified protocol stack for the nRF52840 SoC. The S140 SoftDevice is a Bluetooth Low Energy Central and Peripheral protocol stack solution. The S140 SoftDevice integrates a Bluetooth Low Energy controller and host and provides a flexible API for building Bluetooth Low Energy nRF52X SoC solutions. The development and experiment are based on nRF5_SDK V15.3.0.



BLE uses channels 0-36 for data transmission (light orange in picture below) while channel 37, 38 and 39 are used as advertising channels (pink in picture below).





III – BLE Connection Procedure






If central finds the peripheral with a specific name, it will send a connection request to the peripheral. After they are connected, the GATT client and server will exchange parameters including ATT_MTU, Data Length Extension and PHY rate.



IV – GATT Parameters Setting

ATT_MTU size:
The Attribute Maximum Transmission Unit (ATT_MTU) defines the amount of data a device can send/receive per GATT operation. The default MTU is 23 bytes and can be increased to 247 bytes. When increasing this value, longer payloads can be achieved by sending several packets in one transaction. In order to maximize the use of the Data Length Extension, the MTU should always be set to (Maximum Data Length – 4). This means that for the maximum data length of 251, the optimum MTU to set is 247 bytes.

Data Length Extension (DLE):
The data length extension feature allows the LE controller to send packet data units (PDUs) with payloads of up to 251 bytes. Data length extension uses larger packets so that more data can be sent in one packet, increasing throughput. The nRF52840 supports the increase of the data packet length from the default 27 to the maximum value allowed by the Bluetooth spec which is 251 bytes.

Physical layer (PHY) data rate:
The over-the-air data rate used to be limited to 1Mbps in BLE. However, Bluetooth 5 has higher data rates to achieve faster transmission and uses coded PHY for long range transmissions.

Connection interval:
A BLE connection interval is the time between two data transfer events (connection events) between the central and the peripheral device. When increasing this value, more packets may be sent in one interval but if a packet is lost, the wait until the re-transmission is longer. Increasing this value can increase throughput, provided that the GAP event length increases by the same amount or the connection event length extension is enabled.







V – Throughput Test

Test case-1: Different PHY rate

Increasing PHY rate results in throughput improvement.



Test case-2: Different ATT_MTU

Increasing ATT_MTU results in throughput improvement.



Test case-3: DLE enable/disable

Enable Data Length Extension (DLE) will increase throughput.



Test case-4: Different connection interval

Increasing connection interval result in throughput improvement.



Test case-5: Different connection interval—one device (client) is shaded partially to simulate the packets lost.

This test case aims to simulate the length of time between packet loss and re-transmission. This delay will cause slower throughput. When the value of connection interval is longer, the throughput is worse. The level to set the connection interval depends on the packet loss and the environmental effect.



Optimizing for maximum data throughput

Always enable DLE
– If you’re using Bluetooth v4.1 or earlier, this is not a valid option.

Use LE 2M PHY
– If you know that the devices on both ends support BLE 5, choosing the LE 2M PHY is one of the best ways to maximize your application data throughput.

Use maximum ATT MTU value 247 bytes.

Connection interval parameter depends on the testing environment. Chose maximum interval time if there is no packet loss.



Download PDF >





See all Technical Papers >


Enterprise AP – Piggyback information through DHCP
Ryan Hsu  |  EnGenius Technologies







I – Introduction

We use the framework of the Dynamic Host Configuration Protocol (DHCP) to pass configuration information to the host in layer 3 network equipment. As processor power gets stronger, we would also want to carry related information in access point. The framework to carry information in IPv4 and IPv6 are quite different. This paper provides the design direction for both.



II – Dhcpv4 Design Consideration

In IPv4, configuration information is carried in tagged data items that are stored in the ‘options’ field of the DHCP message. In layer 3 network equipment, the options can be added by the DHCP relay agent. In an access point device, however, the options can only be added by “tampering” with the packet when the DHCP packet passes through.

In an access point device, the WiFi interface and Ethernet interface are bridged together. To intercept the packet, we can add an iptables rule in the start of the chain (raw, PREROUTING) to make the DHCP packet (UDP, destination port 67) go to nfqueue. In the nfqueue call back function, the packet can be re-packed with options we need and then resent.

Since the modified packets are broadcast packets, it is necessary to filter the same packets propagating from the bridge interface. To filter the same packet, we can create a drop rule in ebtables (filter, FORWARD, UDP, destination port 67).











III – Dhcpv6 Design Consideration

In Internet Protocol Version 6 (IPv6), things get more complex. When the packet passes to the relay agent, the agent does not tamper with the packet but sends to the server a relay-forward packet with the original packet inside. The options are added in the relay-forward packet but not the original packet.

RFC 6221, Lightweight DHCPv6 Relay Agent (LDRA), was submitted for this scenario. In this scenario, the WiFi client connects the access point with the LDRA, the access point (LDRA) forwards the packet to DHCPv6 Relay Agent (DRA), and the DRA forwards the packet to the DHCPv6 Server.

To bring configuration information into the relay forward packet, implementing an LDRA into the access point is necessary. There are several implementations of DRA (dibber-relay or isc-dhcp-relay) allowing us to slightly modify the DRA to LDRA.







Observe the transmit/receive packets of LDRA and DRA in Diagram 3.

Comparing LDRA to DRA, we can find:

1. The receive packets from WiFi client (solicit, advertise, request, reply, release) are exactly the same. (Diagram 2. A.1 ~ A.4, D.1~D.2 and Diagram 3. A.1 ~ A.4, D.1~D.2)

2. The packets between LDRA and DRA, and the packets between DRA and DHCPv6 server have the same format with different destinations. (In Diagram 2, B.1 vs. C.1, B.3 vs. C.3, E.1 vs. F.1.

With DRA source, we can modify these cases, change the destination to multicast address (FF02::1:2).







IV – DHCP Relay Information

RFC 3046 defines DHCP relay information with options that can be carried in the DHCPv4 message. RFC4649 defines options which can map DHCPv4 options. For an enterprise access point, we care about vendor information and interface information. In DHCPv4, the option is 82 (curcuit id, remote id) and in DHCPv6, the options are 37 (remote id), 18 (interface id), and 16 (vendor id). This information can be added in the nfqueue callback when we need them in IPv4 and be added in LDRA when we need them in IPv6.



References

1.RFC 6221 Lightweight DHCPv6 Relay Agent
2.RFC 3046 DHCP Relay Agent Information Option
3.RFC 2132 DHCP Options and BOOTP Vendor Extensions
4.RFC 4649 Dynamic Host Configuration Protocol for IPv6 (DHCPv6) Relay Agent Remote-ID Option
5.RFC 3315 IPv6 Dynamic Host Configuration Protocol for IPv6 (DHCPv6)



Download PDF >





See all Technical Papers >


IPV6 NAT Proxy Implementation
George Lin  |  Senior Software Engineer







ABSTRACT — Due to the unlimited number of IPV6 addresses, IPV6 seems to make NAT unnecessary. Thus, there is no traditional way to execute NAT operation in IPTABLES like IPV4. This article briefly discusses how to implement a simple IPV6 NAT server to redirect packets to a remote server for security check as an application for IPV6 NAT.



I – Introduction

NAT (Network Address Translation) is a method of remapping one IP address into another by modifying the IP header of packets while transmitting across routing devices. In IPV4, different devices in the same subnet can appear as a single device on the internet by using the NAT technique. NAT is a general and essential way to conserve global address space in the face of IPV4 address exhaustion.

And now, the exhaustion problem seems to be solved by the unlimited number of IPV6 address spaces available. Still, the NAT technique is valuable and useful for many IPV6 applications such as security checks in web browsing.



II – IPV6 NAT Proxy

In Linux system, the general way to implement an NAT operation is to use IPTABLES. But it only supports IPV4 until the IPV6 NAT proxy is accomplished by the SENAO network. The IPV6 NAT proxy is a kernel module registered in the netfilter. When the user triggers the HTTP request for web browsing, the IPV4 and IPV6 packets will flow into the netfilter, and the IPV6 NAT proxy will sniffer the IPV6 packet flow. By filtering out the HTTP IPV6 packet, the IPV6 NAT proxy will transfer the packet to a remote server by remapping the destination IPV6 address to the IPV6 address of the security server (see figure).

The remapping procedure includes calculating the TCP checksum of IPV6 packets while completing network address translation. IPV6 NAT proxy keeps the TCP connection tuple until the security server returns the checking result. Once the security server returns the positive result, the original packet will be sent out without the user noticing. Otherwise, the HTTP connection will be dropped by the IPV6 NAT proxy and the user will be blocked for browsing the fraud.

Besides translating to the security server, the IPV6 NAT proxy provides options to set different IPV6 addresses, IPV6 ports, the live time of the tuple and the maximum number of stored tuples for customization.



III – Conclusion

This article describes a simple way to implement an IPV6 NAT server without discussing full-cone NAT, restricted cone NAT, post-restricted cone NAT and symmetric NAT–as well as invokes the idea of using NAT in IPV6.



References

1. Network address translation https://en.wikipedia.org/wiki/Network_address_translation
2. Getting the most out of IPV6 https://blog.paloaltonetworks.com/2015/08/getting-the-mostout-of-ipv6



Download PDF >





See all Technical Papers >


Fisheye Optic Center Calibration
Wei-Siang Wang  |  EnGenius Technologies







ABSTRACT — With the rapid evolution of the IP camera industry, one of the more popular IP cameras is the 360° camera. It enables a complete, surround view of an area while fisheye lenses provide very large wide-angle views. However, the fisheye lens is a wide-angle lens that captures warped images with distorted appearance. The images produced suffer from severe distortion as a result of the warped scene being projected onto the flat scene. It is important that optic parameters be found to relieve the distortion. The method for finding out the optic center and the radius of the sphere is discussed in this paper. Based on experimental results, the optic center and the radius can be found effectively by the proposed scheme.



I – Introduction

The fisheye lens is a wide-angle lens that captures warped image with distorted appearance. Users are also able to flatten or dewarp the image into a rectilinear or panoramic view. The viewing modes available with the chip include:

“O” for “Original” view: This is the original, warped image captured by the camera.

“P” for “Panoramic” view: This is the basic, panoramic view which has been dewarped.

“R” for “Regional” or “Rectilinear” view:
This view allows for a single view, roughly equal to one quadrant of the overall image, which can make use of pan, tilt, or zoom operations using the camera’s PTZ feature.

For example, a common usage of 1O dewarping is shown in Fig.1, the 1O image is dewarped to the 1P image. It is critically important for the chip to obtain the suitable optic parameters for dewarping.

There are three optic parameters in the chip setting: RADIUS, HSHIFT and VSHIFT. We can derive these parameters from the circle position which is fetched by the proposed scheme.



Figure 1: 1O dewarping. (a) 1O mode, (b) 1P mode.









Figure 2: Fisheye image circle. (a) perfect case, (b) practical case.






II – The Proposed Scheme

Based on the above-mentioned situations, in order to fetch thecircle position, there are several stages which are described as follows:

A. Generate an image that has a clear boundary in 1O mode

In order to generate an image that has a clear boundary in 1O mode, we can cover the camera lens with a semi-opaque mask. It is noted that we need to provide enough light source in the top of semi-opaque mask. Fig.3 exhibits a simulated installation for this stage.



Figure 3: Cover the camera lens with a semiopaque mask for simulate installation.






As can be seen in Fig. 4, we obtained an image that has a clear boundary in 1O mode. Using this feature, we can select a suitable threshold to detect the circle boundary.



Figure 4: An image that has a clear boundary in 1O mode.






As can be seen in Fig.5, we need to obtain the coordinates of point a and point b so that we can calculate the center of the circle.



Figure 5: The position of point a and point b.






B. Smoothing the target region

The Gaussian smoothing operator [1] is a 2-D convolution operator that is used to blur images and remove detail and noise. Fig.6 shows a suitable integer-valued convolution kernel that approximates a Gaussian with standard deviation of the distribution = 1. After revealing the unadulterated form of the pixel, we can further improve the image processing effect and reduce false positives.



Figure 6: Discrete approximation to Gaussian function with standard deviation of the distribution = 1.






As can be seen in Fig.7, we set up the regions where we want to search for the boundary point. The smoothing process can only be applied to these regions.



Figure 7: The regions that we want to search the boundary point.






C. Search the boundary point

The resolution of the image is 640×480, the distance between the circle boundary point and image boundary is quite different in horizontal and vertical situations. We can set a proper offset to address this issue. As can be seen in Fig. 8, we obtain the pixel value by raster scan. If the current pixel value is greater than the selected threshold value (i.e., brighter than the threshold), the current position can act as boundary point.



Figure 8: The search direction in each region.






The coordinates of the point a and point b can be retrieved from the following regions:

Region A → ay

Region B → ax

Region C → bx

Region D → by

Once we get the position of point a and point b, the center of circle and the radius can be calculated.

D. Writing the parameters to flash memory

Owing to the requirement of the chip parameter during the boot process, we need to execute this program in the manufacturing process. When the parameters are fetched from this program, we can write the parameters to flash memory that can be prepared for further usage (e.g., chip initial process and web UI).



III – Experimental Results

The proposed scheme has been implemented in Linux platform. Fig. 9 illustrates four different devices; the circumference of circle and the center of circle are shown in black color, respectively.



Figure 9: A visual presentation in four different devices. (a) device A, O(309, 234), radius = 221, (b) device B, O(321, 244), radius = 224, (c) device C, O(316, 250), radius = 222, (d) device D, O(320, 244), radius = 221.






IV – Conclusion

In this paper, we propose a method for finding out the optic center and the radius of the sphere. Based on experimental results, the optic center and the radius can be found effectively by the proposed scheme.



References

1. R. C. Gonzalez and R. E. Woods, Digital Image Processing, 3rd ed., Prentice Hall, 2007.



Download PDF >





See all Technical Papers >


Dynamic DNS Service with Port Redirection
Yuan-Ping Chang  |  Section Manager







ABSTRACT — The dynamic DNS service with port redirection technology will direct the connection to its own dynamic DNS server, determine the IP address and port that the user wants to connect to, and direct the connection to the correct address. External users will not be aware of the use or if there have been any changes in behavior.



I – Introduction

DNS (Domain Name Service) is a service on the Internet that connects Domain Names to IP addresses, allowing users to easily access the Internet without having to remember IP addresses.

Each host is assigned an IP address when it is connected to the Internet to transmit data, but this IP address will be dynamic in most cases. When the host leaves the Internet and reconnects, your ISP will randomly assign a different IP address. In this case, the Domain Name and the dynamic IP address cannot correspond to each other, so dynamic DNS technology is used to solve this problem.

The dynamic DNS service can point the Domain Name to a host that may change the IP address frequently. When the host finds that its IP address has changed, it sends a request to the dynamic DNS server to inform the dynamic DNS server of its newly acquired IP address. The dynamic DNS server instantly updates the IP corresponding to the host’s Domain Name to other DNS servers. When the user wants to connect to the host, the host’s IP address is obtained through DNS all over the world. Finally, the external user can connect to the host of the dynamic IP address user through the Domain Name.


II – Background

In order to prevent personal or small business users from providing services such as WEB and FTP, many ISPs block these external ports such as port 80, which is dedicated to HTTP, so that users cannot set up their own websites on their home network. Therefore, users can only provide web services by using other ports. This causes a lot of inconvenience to people who want to browse the webpage for them. People must specify a specific port in their browser to use web services.

The dynamic DNS service only converts the Domain Name to an IP address, which doesn’t address the problem that the common port has been blocked by the ISP. Therefore, we need a new dynamic DNS mechanism to solve such problems.

When the common external port is not available, we use dynamic DNS service with port redirection technology to allow the HTTP service server to direct the external user’s connection to the correct port through the Port Redirect dynamic DNS server.


III – How Dynamic DNS Service With Port Redirection Works

Unlike the traditional dynamic DNS server, the port redirect dynamic DNS server allows the HTTP server to specify the IP address and port to be translated. The external user can use the port redirect dynamic DNS server to convert the connection from the Domain Name to the IP address and port specified by the HTTP server.

For example, the HTTP server provides a service port of 10000, and specifies http://DomainName to http://SelfIP:10000 to the port redirect dynamic DNS server. When people want to connect to the HTTP server, the IP address obtained from DNS will be the IP address of the port redirect dynamic DNS server. Then the port redirect dynamic DNS server will redirect this connection to the IP address registered by the HTTP server, allowing people to successfully connect to the services provided by the HTTP server.

During the above process, people who need to access the HTTP server do not need to adjust any settings or behavior; the port redirect dynamic DNS server will automatically guide and transfer, so that the HTTP server can be accessed when the common port is blocked by the ISP. This is illustrated in Fig.1.



Figure 1: Dynamic DNS Service with Port Redirection Workflow






1. Users set mesh network settings include mesh-ID, channel and 1. Device update its IP, internal port and external port to port redirect dynamic DNS server.

2. Port redirect dynamic DNS server will sync DNS records with worldwide DNS server.

3. People query the Domain Name of device. The DNS server response the IP address of port redirect dynamic DNS server.

4. People connect to port redirect dynamic DNS server.

5. Port redirect dynamic DNS server redirect the connection to the device.
IE and decrypt these information to establish a temporary connection with mesh router.

6. The mesh router produces a candidate list of nodes through the scanning result and WDS temporary connection. APP or GUI gets the list, and user applies the wanted nodes from the list.

7. The mesh router would send authentication information to candidate nodes through temporary connections.

8. The new or un-configured devices get authentication information to establish a formal connection with Mesh router and set itself be configured.


IV – Conclusion

Through dynamic DNS service with port redirection technology, the HTTP service server can provide services smoothly even if some ports are blocked by the ISP.

People can enter their Domain Name on the Internet to direct the HTTP service server via the Port Redirect dynamic DNS server.



Download PDF >





See all Technical Papers >


Auto Timezone Detection
Steven Lin  |  Senior Software Engineer







ABSTRACT — To record data and operate normally, devices must display the accurate time for different countries and regions. In order to solve this problem, we have implemented a mechanism to automatically detect the time zone, allowing the device to get the time zone that corresponds to the current network location.



I – Introduction

The time zone is the same time definition of the region on Earth. In 1863, the concept of time zone was first used. The time zone partially solves this problem by setting a standard time for the region.

The machines on the network basically use NTP (Network Time Protocol) to synchronize their clocks over the Internet. There are many NTP servers on the network that can synchronize the correct time.


II – How To Get Timezone

In order to synchronize the correct time with the NTP server, providing a corresponding time zone is the first necessary step. To determine a time zone, we must have latitude and longitude.

We use the IP (Internet Protocol) address as an anchor point to obtain latitude and longitude, which can then help us determine the correct time zone.



III – Manual To Automatic

Before we can implement the automatic time zone detection mechanism, we must manually set or select the time zone of the device through the GUI. While it is inconvenient, setting the time manually will allow the device to automatically get the time, removing the need to set it manually. Since it is fastest and most convenient for the device itself to automatically query the time zone via the network, we implemented a mechanism to automatically detect the time zone.


IV – Timezone Detection Mechanism

Because some methods on the network achieve only partial results, we have developed a complete detection mechanism that requires only three conditions:

1. List of global time zones

2. IP address

3. The server that can check the time zone after giving latitude and longitude

The first two pieces of information can be provided through prepreparation and the device itself. There are many servers on the Internet that provide time zone check services. We can get results by providing the corresponding information to the server. Then update the acquired time zone information to the device, and the implementation of this method is completed.


V – Conclusion

The way to set the time zone only through the GUI can also be replaced by automatic acquisition. With this automatic detection time zone mechanism, the device can automatically get the correct time. It is more convenient for the data record and system operation.


References

1. Ntppool.org. 2020. Pool.Ntp.Org: The Internet Cluster Of Ntp Servers.
2. Google Developers. 2020. Public NTP|Google Developers.



Download PDF >





See all Technical Papers >


The Auto Negotiation Service Between Router and IP Camera Based on UPnPG
Yuan-Ping Chang  |  Section Manager







ABSTRACT — As IP cameras have become more and more common, so has the problem of managing a numerous network of IP cameras. In this paper, we will introduce a solution to users who have many IP cameras running at the same time. Our system will allow them to easily manage their IP cameras.



I – Introduction

IP camera (Internet Protocol camera) is a networked digital video camera that receives control data and sends image data over a local area network or the internet. The IP camera, like any network device, is assigned an IP address when it is connected to your router and powered on. Some of them require a network cable connection and some are wireless and transmit their packets via radio frequency signals over the Wi-Fi network. Users can access the IP camera by typing in a specific IP address to a web browser, a video streaming player, or mobile application. They can then watch the camera live stream, receive push alerts, check the camera recordings, and configure the IP camera settings wherever they are.


II – Background

Connecting an IP camera from your computer is easy. But the question now is, how do you manage multiple IP cameras from your computer?

The challenge is that when your IP cameras increase, you need to find a way to manage them effectively. You can set port forwarding rules on the router for your IP cameras one by one, or enable the UPnP feature on your router. When you need to access your IP cameras, it’s not easy to remember each of their IP addresses. And it should not be a problem to easily access their configuration webpage or streaming video.

In this paper, we propose a solution called “The Auto Negotiation Service between Router and IP Camera Based on UPnP” that can display all IP camera information on the router webpage. Users do not need to write down the IP address of each IP camera. They just need to type the router’s IP address into a web browser.

They can know the IP address, MAC address and device name of all IP cameras connected to this router.



III – How Auto Negotiation Works

This Auto Negotiation Service between Router and IP Camera feature is based on UPnP technology. Users can connect to the web page of the IP camera by clicking the IP camera list directly on the router. This is illustrated in Figure 1.



Figure 1: IP Camera List



Figure 2: The router side works


A. Router side works

Once the router receives the UPnP probe packet, it is recognized as an IP camera that supports the auto negotiation feature by looking up the UPnP description field. If the IP camera supports the auto negotiation feature, the router will update its IP camera list. This is illustrated in Fig.2.

1. IP camera sends UPnP probe with specific description.

2. Port forwarding rule is created by UPnP, and the router updates its own IP camera list.

3. User accesses the webpage of the router to get the IP camera list.



B. IP camera side works

The description field of UPnP probe packets contain the device name, MAC address, HTTP port, RTSP port, RTCP port, etc.

The IP camera sends out UPnP probe with specific description when power is on. When the setting of the IP camera is being changed, the IP cameras will send out UPnP probes again to update their information on the router. Make sure that the latest information of the IP camera is on the router.



Download PDF >





See all Technical Papers >


Dual Image on EnGenius Designed ARM-based Product Families
Chin-Ting Chu  |  Software Engineer







ABSTRACT — This document contains information on the changes required to support dual images in the bootloader level. Dual image is one workable idea to provide alternating firmware between two firmware partitions. When the current firmware is broken, we reset the kernel entry and boot it up. Most importantly, we do the firmware upgrade in the inactive partition. If there is nothing wrong with it, the upgrade process will swap the kernel entry. Therefore, we can keep one bootable firmware in our device all the time.



I – Introduction

The bootloader does some low-level hardware initialization, such as processor, memory, ethernet, flash, PCI, UART, and so on. After setting system configurations, the bootloader loads the kernel and passes startup information including serial port speeds, clock rates, the table of partitions and other hardware configuration data.

When the kernel is loaded, it configures the hardware, allocates the memory, loads the drivers, inserts the modules, mounts the root filesystem, and runs the pre-init processes. The final step is spawning the init process, the first user space process. The three types on the ARM-based OpenWRT are NOR-only, NAND-only and NOR plus-NAND.



Figure 1: NOR only, NAND only, NOR and NAND






II – Procedure

I. NOR flash only



Figure 2: Use the 1st firmware – NOR only






1. Bootloader : When the power on, the bootloader will check the dual image flag, “active_fw”

1.1. Append additional partitions to mtdparts : kernel_1 and rootfs_1
1.2. Load the target kernel to the memory
1.3. Assign a active root filesystem partition by setting the bootargs.
1.3.1. If “active_fw=0” => root=rootfs
1.3.2. If “active_fw=1” => root=rootfs_1
1.4. If boot kernel fail, swap active firmware partition

2. Kernel Space : parses the bootargs assigned by bootloader, and mount the root filesystem.

3. User space : when system upgrade

3.1. If “active_fw=1” =>
3.1.1. flash kernel and rootfs
3.1.2. set active_fw=0
3.1.3. reboot
3.2. If “active_fw=0” =>
3.2.1. flash kernel_1 and rootfs_1
3.2.2. set active_fw=0
3.2.3. reboot

II.. NAND flash only
The procedure is the same as ‘NOR flash only’.

III.. NOR + NAND



Figure 3: Use the 2nd firmware – NOR+NAND






1. Bootloader : When the power on, the bootloader will check the dual image flag, “active_fw”

1.1. Append two NAND partitions named rootfs and fs2
1.1.1 If “active_fw=0” =>

The upper half of NAND is named rootfs and the another one is named fs2.

1.1.2 If “active_fw=1” =>

The upper half of NAND is named fs2 and the another one is named rootfs.

1.2. Load the target kernel to the memory
1.3. If boot kernel fail, swap active firmware partition

2. Kernel Space : Under the NOR+NAND structure, we always use the partition named rootfs as the root filesystem.

3. User space : when system upgrade
The procedure is the same as ‘NOR flash only’.



lll – Implement

MTD partitions: two equal parts for two firm-wares

II. Dual image flag: set a active firmware partition

III. System upgrade: Always upgrade the inactive firmware partition

IV. Auto recovery: If the uboot finds the boot fail from this firmware partition, active the another firmware partition.


A. NOR flash only

Figure 4: MTD partition table – NOR only






1. Create two firmware partitions

1.1. modify the nor-partition.xml to tell the bootloader the new partitions info
1.2. modify the sys_flash_map.mk to tell the kernel mtd partition table

2. Dual image flag

2.1. Modify board.c to add “active_fw” to uboot de-fault env
2.2. Modify the cmd_bootipq.c to swap the boot args(root= rootfs or root= rootfs_1).

3. System upgrade

3.1. check “active_fw” and flash the target firmware [email protected]
3.1.1. “active_fw=1” => flash kernel and rootfs
3.1.2. “active_fw=0” => flash kernel_1 and rootfs_1

4. Auto recovery

4.1. Modify the board.c to make sure the uboot catch-es the return value from the boot function

B. NAND flash only

The procedure is the same as ‘NOR flash only’.



C. NOR + NAND

Figure 5: MTD partition table when active_fw=0 – NOR+NAND






1. Create two firmware partitions

1.1. Modify bootipq.c to add a partitions named fs2
1.2. Modify bootm.c to pass the atags partition info. to kernel

2. Dual image flag

2.1. add “active_fw” to uboot default env

3. System upgrade

3.1. check “active_fw” and flash the target firmware [email protected]
3.1.1. Flash the fs2 partition

4. Auto recovery

4.1. Modify the board.c to make sure the uboot catch-es the return value from the boot function



Download PDF >





See all Technical Papers >


Simple Configuration and Connection with Authentication for Wireless Mesh Network
Leonard Chiang  |  EnGenius Technologies







ABSTRACT — Wireless Mesh Networks have become popular in recent years. However, setting up multiple devices is a complex problem. In this document, we proposed a simple solution called Mesh Simple Configuration with Authentication (MCSA) to configure mesh nodes more efficiently.



I – Introduction

Wireless mesh networks for local area networking have become popular in recent years in part because they solve some of the shortcomings of single router WLAN networks. Wireless mesh networks contain multiple nodes—routers and/or access points—working in concert to deliver data within a network. Multiple nodes can be placed strategically within an office or home to eliminate dead spots and to ensure that the signal strength is adequate wherever the LAN is needed. That is, a mesh network can extend the range and coverage area otherwise achieved by a single wireless node. A mesh network can also be more reliable than a conventional network configuration by providing redundant paths for data traffic, which can allow for uninterrupted communications even if a node fails.


II – Problem Statement

Existing wireless mesh networks have not been easy for most consumers to set up. Traditionally, setting up a wireless mesh network or adding nodes to an existing wireless mesh network have required complicated configuration of numerous parameters.

One of the most challenging aspects in setting up wireless mesh networks was that each node needed to be configured individually. This meant that the more nodes that one wished to add to a wireless mesh network, the more laborious the configuration one had to undertake.

A need therefore exists for a simple way to configure a wireless mesh network, particularly to simultaneously add multiple nodes to a wireless mesh network.


III – Solution

In order to solve the above-mentioned problems, the solution requires systems and methods for the simple configuration of wireless mesh networks, in particular the addition of nodes to such networks.

In this solution, new devices in the vicinity of a wireless mesh network broadcast signals, such as beacons, advertise their ability to join the wireless mesh network. These beacons may be detected by a primary device, such as a primary router, that is part of and can manage the wireless mesh network. The primary device is then able to establish temporary connections with the new devices that are eligible to join the wireless mesh network.


IV – Descriptions

1. Users determine mesh network settings including mesh-ID, channel, and password at the mesh router by mobile app or graphical user interface (GUI).

2. After the un-configured device powers on, in the first 6 minutes, the device creates a temporary interface to broadcast WDS learning IE and listens to receive the particular beacon with MSCA IE.

3. Users can trigger mesh simple configuration by app or GUI. The mesh router does a scan first to collect any neighbor’s beacon with WDS learning IE, and then creates a temporary interface to broadcast the beacon with MSCA IE.

4. Un-configured devices receive the particular beacon with MSCA IE and decrypts this information to establish a temporary connection with the mesh router.

5. The mesh router produces a candidate list of nodes through the scanning results and WDS temporary connection. The app or GUI gets the list, and the user applies the desired nodes from the list.

6. The mesh router sends authentication information to the candidate nodes through temporary connections.

7. The new or un-configured devices get authentication information to establish a formal connection with the mesh router and sets itself to be configured.



Figure 1: Dynamic DNS Service with Port Redirection Workflow






V – Conclusion

Systems and methods for managing a wireless mesh network, in particular to provide for the simple configuration of a plurality of unconfigured devices to be added to the wireless mesh network are proposed. New devices, such as beacons, in the vicinity of a wireless mesh network broadcast signals, advertising their ability to join the wireless mesh network. These beacons may be detected by a primary device, such as a primary router, that is part of and can manage the wireless mesh network. The primary device is then able to establish temporary connections with the new devices that are eligible to join the wireless mesh network. The primary device may generate a list of potential new nodes for presentation to an electronic device, such as a smartphone, from which a user can select a plurality of new devices to add as new nodes. Alternatively, the primary device automatically adds devices as nodes to the wireless mesh network based on various criteria.



Download PDF >





See all Technical Papers >