Design and Implementation of MPEG-2 TS Stream Analysis Module and System

This article refers to the address: http://

Abstract : This paper introduces the design and implementation of a general analysis module and system for PC-based MPEG-2 and DVB transport streams. The analysis module enables analysis of various information in the transport stream and testing of ETR290 parameters. The module fully applies the design idea of ​​multi-thread scheduling, modularization, etc., so that the module can cooperate with various acquisition data modules such as hardware boards to complete the function of transport stream analysis. Applying this module to implement a static transport stream analysis system, verifying the function of the analysis module.
Keywords : MPEG-2, DVB, TS, multi-thread scheduling, modular design

MPEG-2 is an international standard for moving image compression, in which the transport stream has become a system-level coding standard commonly used in the field of digital television. The transport stream can be used to combine multiple channels into one stream for transmission with a certain error channel. It allows the same program to contain multiple audio and video streams, and can also add some additional information. The transport stream has a fixed packet length of 188 bytes. Its structure is extremely complicated. For the user, for a specific stream, it is first necessary to know which PID (Packet ID) packets it consists of. MPEG-2 defines a table of information used to describe the content carried by the transport stream, called the Program Specific Information (PSI) table, which includes four tables associated with it. Among them, the program association table PAT and the program map table PMT are the two most critical tables for determining the content of each program in the current transport stream.

The DVB standard currently adopted in China is a digital television standard adopted by European countries. The baseband data format adopts the MPEG-2 standard, that is, video and audio coding and code stream multiplexing are in compliance with the MPEG-2 standard. The DVB standard extends the PSI and is collectively referred to as service information, that is, SI. This information, as well as the normative nature of the transport stream, is critical to the correct receipt of digital television programming.

The integration of digital TV systems is a huge and complex project. In order to quickly and effectively find fault sources and solve problems in development, it is often necessary to analyze digital TV code streams. In addition, the application of code stream testing technology is more widely used in the monitoring of digital TV networks. Since TS is a small correlation between compressed signal data, and the TS stream has a strict data format, small errors will result in a sharp drop in image quality or even decoding, so the signal must be monitored. To this end, we have developed this universal transport stream analysis module, which can analyze various code stream related information including program information, code rate, bandwidth allocation, PCR accuracy and buffer overflow, and also can carry out ETR290 related parameters. The test, because the design is fully adopted the idea of ​​modular design, the module can work together with various modules to obtain data to complete the transport stream analysis in different environments. In addition, in order to verify the function of the analysis module, we developed a static analysis system for TS files. The following focuses on the design and implementation of the analysis module and the application of the module in the system.

1. Design thinking
Functionally, the entire analysis module completes the analysis of the various information of the code stream and the testing of the ETR290 parameters. The analysis of the code stream information includes: basic information, program information, bandwidth information, audio and video information, multiplexing structure, PCR information, and buffer information. The basic information includes basic information such as the rate of the stream, the stream ID, and the name of the provider; the program information includes the name of each program in the transport stream, the PID (Packet ID) allocation, the program rate, the program list, and the like; the bandwidth information includes The bandwidth occupied by each PID; the audio and video information contains various information related to the audio and video in the stream; the multiplexing structure shows the multiplexing of each program in the stream; the PCR information shows the accuracy and interval of each PCR in the stream; The buffer information contains the state of each buffer when the virtual decoder (STD) decodes the current stream. In addition to this information, the module completes the testing of the ETR290 parameters, which are the parameters of a series of tests given by the DVB standard in the official documentation. The general goal is to provide the most important elements in the TS stream. Health test", according to the importance and different nature, the parameters of these recommended tests are divided into three priority levels, wherein: the first priority is a set of basic parameters, which ensure that the ts stream can be decoded; The two priorities focus on additional parameters that are recommended for continuity detection; the third priority concentrates on parameters that may be of interest to a particular application.

The module is designed to maintain versatility, that is, as long as the module can obtain the data of the transport stream through the interface, the analysis can be completed. In practice, a module for acquiring data from the file is designed to test the working condition of the analysis module. Both the module and the analysis module are designed in the form of a dynamic link library, ie, a dll. At the same time, an upper application for displaying the result is written. The program combines the acquisition data and the analysis module to implement a complete transport stream static analysis system. It can be seen that if the analysis module needs to be used for the real-time analysis system, as long as the corresponding data acquisition device, such as a hardware data acquisition card, the dll providing the same function is used to analyze the module to obtain data. In addition, strong scalability in design is also one of the goals, and I hope that in the future, more analysis functions can be added flexibly and conveniently as needed. The design and implementation of the analysis module and its position and function in the actual system will be described below using the above static analysis system as a carrier.

2. Design and implementation
2.1 Overall design
As shown in Figure 1, the whole system is divided into three large parts, namely, a data module, an analysis module, and an interface module.

The data acquisition module needs to perform preprocessing of the code stream after obtaining the data from the data source (which may be a file, or may be a real-time input transport stream, or a TS file in a static analysis system), and the code stream preprocessing function is Each packet in the code stream is tagged with a time stamp that provides the arrival time of each packet during subsequent analysis, thereby facilitating testing. For the real-time stream, the data acquisition card of the hardware can generally be added to the actual arrival time of the corresponding packet, and for the data obtained from the file, the time label corresponding to the packet can be calculated according to the PCR in the stream. It can be seen that no matter what form of data acquisition device, as long as it can provide the above functions, the analysis module can work normally.

The analysis module includes a circular buffer and the actual data analysis unit, which will be described in detail later.

The interface module is used for information exchange between the system and the user. The user control part is responsible for conveying the user's commands to the system, such as starting analysis, stopping analysis and selecting files, etc., and the result display part displays the results of the transport stream analysis and test to the user.

2.2 Application of multi-thread scheduling technology If the entire analysis module adopts single-threading, that is, each time a certain amount of data is acquired and then analyzed, so that the system resources are not fully utilized, the efficiency of the analysis and testing will inevitably be caused. This view, in the early days of development, has been tested using this single-threaded approach. The results show that even in this case, even a sub-module of basic information analysis will make the efficiency of the analysis module completely complete when processing real-time streams. Unacceptable. To this end, a circular buffer is set up as a relay station for data, as shown in Figure 2.

As shown in the figure, thread 1 writes data from the data acquisition module and writes it to the circular buffer; thread 2 reads the data from the circular buffer, and then analyzes it with the analysis unit. The circular buffer exists in the form of a class object, and attention is paid in programming. The problem of synchronization between threads. After applying the multi-thread scheduling technology, the efficiency of the analysis module is greatly improved. The application of this technology lays a good foundation for the implementation of analysis and testing.

2.3 Analysis unit design The analysis unit is the core of the whole module, and performs two parts of work in function: analysis of code stream information and testing of ETR290 parameters. The analysis of code stream information mainly corresponds to the semantic analysis of code stream. The work of this part corresponds to several small modules according to the principle of modularization. Each module completes the analysis and extraction of one kind of information; the ETR290 parameter test mainly corresponds to Grammatical tests, as mentioned earlier, the ETR290 parameters include three priority levels, which can be divided into four broad categories according to the test method: SI table related tests, PCR related tests, buffer tests, and other tests. The buffer test is taken as an example for a more detailed explanation.

The MPEG-2 standard specifies a virtual decoder STD (System Target Decoder). STD is a conceptual model that can be used to specify the decoding process in constructing and verifying TS streams, although different decoders can have different Structure, but they must ensure that a TS stream that can be correctly decoded on the STD must be correctly decoded in practice. That is to say, the STD virtual decoder is an important criterion for verifying whether the TS stream can be correctly decoded. ETR290 parameters All the tests related to the buffer are given by the STD related tests, which include three small items, namely buffer errors, empty buffer errors and data delay errors.

The main test method for this type of error is to simulate the work of a decoder according to the rules determined by the standard, so that the transport stream data is transmitted between the buffers of the decoder, including the conversion between the TS packet, the PES packet and the ES packet, and each The analysis of the buffer size and the extraction calculation of the decoding unit of the display unit and the like. Based on the above simulation, three types of errors are tested, that is, whether there is a buffer overflow, whether there is data staying in the STD for too long, and whether some buffers are cleared within a specified time. The analysis of the buffer also exists in the form of a class object. In the object, the transport stream is unpacked according to the rules determined by the standard, the packet is converted, the time tag is extracted, and the like, and the above three parameters are recorded and tested on the basis of the simulation. This section covers almost all of the data processing work in actual decoding and is therefore the most complex of all the analysis test submodules.

In summary, the flow chart of the entire analysis unit is shown in Figure 3.

2.4 Modularity to maintain versatility and scalability As mentioned above, in order to maintain the versatility of the analysis module, the module is designed in the form of a dynamic link library dll, which provides all the interfaces required by the user to complete such initialization. Start the analysis, stop the analysis, take various analysis results, etc., so that the user only needs to understand each interface and complete the writing of the upper application, then the entire code stream analysis system can be completed. In order to analyze the module's downward versatility, the module only needs to complete the data pre-processing work and provide the data for the module that acquires the data. The actual dll used to obtain the data from the file is an example. The developed analyzer of the usb interface can replace the current dll that obtains data from the file as long as the hardware driver can implement the same pre-processing and data acquisition functions, thereby completing the development of the entire system of the code stream analyzer of the usb interface.

In the process design of the analysis module, the idea of ​​modularity is also applied everywhere. Since the data of the transport stream is an arrangement of fixed-length packets, the functions of various analysis are designed as separate modules that receive a series of packages. The design enables development and testing in stages in development. For example, the program information analysis module can be developed and tested separately. Once integrated, it can be integrated into the analysis thread. This structure provides great support for testing and adding new analysis functions in the future. Convenience.

2.5 Module internal and external communication
Since there may be errors in the analysis process, the user of the analysis module also needs to control which analysis and test sub-modules are started. Therefore, there must be a communication mechanism that enables the analysis module to exchange information with the outside world. At the time of initialization, the user can specify the sub-module of the analysis or test to be started through the interface; in addition, the message mechanism is used to notify the user of the analysis module when the analysis and the test find an error, and then the end user is prompted accordingly.

3. Conclusion
In the development, the above static analysis system was used to test various types of TS streams such as multi-program, single program, high definition, standard definition, satellite program, HDTV program, etc. The result shows that the module works normally and the result is correct, and at the same time, due to the code of the passed file The rate and size can be used to know the real-time duration of the transport stream corresponding to the file. Therefore, by analyzing the time taken to complete the static test, the working efficiency of the module for real-time analysis can also be known. Experiments show that the module has better performance. Efficiency, can be used in various occasions such as static and real-time analysis, with good versatility and scalability. At present, the analysis system using the analysis module has been applied to the auxiliary development of laboratory digital television products and has played a significant role.

References ( 1) International Organization for Standardization and International Electrotechnical Commission, ISO/IEC 13818-1: Information technology-generic coding of moving pictures and associated audio information: Systems. 1994.
(2) International Organization for Standardization and International Electrotechnical Commission, ISO/IEC 13818-2: Information technology-generic coding of moving pictures and associated audio information: Video. 1994.
(3) International Organization for Standardization and International Electrotechnical Commission, ISO/IEC 13818-3: Information technology-generic coding of moving pictures and associated audio information: Audio. 1994.
(4) European Telecommunication Standards Institute. ETSI EN 300 468: Specification for Service Information (SI) in DVB Systems. 2000.
(5) European Telecommunication Standards Institute. ETSI TR 101 290: Measurement guidelines for DVB systems. 2001.