GSM 03.40
GSM 03.40 or 3GPP TS 23.040[1] is a mobile telephony standard describing the format of the Transfer Protocol Data Units (TPDU) of the Short Message Transfer Protocol (SM-TP) used in the GSM networks to carry Short Messages. This format is used throughout the whole transfer of the message in the GSM mobile network. In contrast, application servers use different protocols, like Short Message Peer-to-Peer or Universal Computer Protocol, to exchange messages between them and the Short Message Service Center (SMSC).
GSM 03.40 is the original name of the standard. Since 1999 it is being developed by the 3GPP under the name 3GPP TS 23.040. However, the original name is often used to refer even to the 3GPP document.
Usage
The GSM 03.40 TPDUs are used to carry messages between the Mobile Station (MS) and Mobile Switching Centre (MSC) using the Short Message Relay Protocol (SM-RP),[2] while between MSC and Short Message Service Centre (SMSC) the TPDUs are carried as a parameter of a Mobile Application Part (MAP)[3] package.[4]
In emerging networks which use IP Multimedia Subsystem (IMS) are Short Messages carried in MESSAGE command of Session Initiation Protocol (SIP). Even in these IP-based networks an option exists which (due to compatibility reasons) defines transfer of Short Messages in the GSM 03.40 format embedded in 3GPP 24.011 as Content-Type: application/vnd.3gpp.sms.[5][6]
TPDU Types
GSM 03.40 defines six types of messages, which are distinguished by the message direction and the two least significant bits in the first octet of SM-TP message (the TP-MTI field):
| TP-MTI | direction | message type |
|---|---|---|
| 0 0 | MS → SC | SMS-DELIVER-REPORT |
| 0 0 | SC → MS | SMS-DELIVER |
| 0 1 | MS → SC | SMS-SUBMIT |
| 0 1 | SC → MS | SMS-SUBMIT-REPORT |
| 1 0 | MS → SC | SMS-COMMAND |
| 1 0 | SC → MS | SMS-STATUS-REPORT |
| 1 1 | any | Reserved |
SMS-SUBMIT is used to submit a short message from a mobile phone (Mobile Station, MS) to a short message service centre (SMSC, SC).
SMS-SUBMIT-REPORT is an acknowledgement to the SMS-SUBMIT; a success means that the message was stored (buffered) in the SMSC, a failure means that the message was rejected by the SMSC.
SMS-COMMAND may be used to query for a message buffered in the SMSC, to modify its parameters or to delete it.
SMS-DELIVER is used to deliver a message from SMSC to a mobile phone. The acknowledgement returned by the mobile phone may optionally contain a SMS-DELIVER-REPORT. When home routing applies, SMS-DELIVER is used to submit messages from an SMSC to another one.
SMS-STATUS-REPORT may be sent by the SMSC to inform the originating mobile phone about the final outcome of the message delivery or to reply to a SMS-COMMAND.
TPDU Fields
The fields of SM-TP messages, including their order and size, are summarized in the following table, where M means a mandatory field, O an optional field, E is used for fields which are mandatory in negative responses (RP-ERR) and not present in positive responses (RP-ACK), x is a field present elsewhere:
| SMS-COMMAND | size | Field name | ||||||
|---|---|---|---|---|---|---|---|---|
| SMS-STATUS-REPORT | ||||||||
| SMS-SUBMIT-REPORT | ||||||||
| SMS-SUBMIT | ||||||||
| SMS-DELIVER-REPORT | ||||||||
| SMS-DELIVER | ||||||||
| field | ||||||||
| TP-MTI | M | M | M | M | M | M | 2 bits | Message Type Indicator |
| TP-MMS | M | M | 1 bit | More Messages to Send | ||||
| TP-RD | M | Reject Duplicates | ||||||
| TP-LP | O | O | 1 bit/ 2 bits |
Loop Prevention | ||||
| TP-VPF | M | Validity Period Format | ||||||
| TP-SRI | O | 1 bit | Status Report Indication | |||||
| TP-SRR | O | O | Status Report Request | |||||
| TP-SRQ | M | Status Report Qualifier | ||||||
| TP-UDHI | O | O | O | O | O | O | 1 bit | User Data Header Indicator |
| TP-RP | M | M | 1 bit | Reply Path | ||||
| TP-FCS | E | E | 1 octet | Failure Cause | ||||
| TP-MR | M | M | M | 1 octet | Message Reference | |||
| TP-DA | M | x | 2–12 octets | Destination Address | ||||
| TP-OA | M | 2–12 octets | Originating Address | |||||
| TP-RA | M | 2–12 octets | Recipient Address | |||||
| TP-SCTS | x | x | M | 7 octets | Service Centre Time Stamp | |||
| TP-DT | M | 7 octets | Discharge Time | |||||
| TP-ST | M | 1 octet | Status | |||||
| TP-PI | M | M | O | 1 octet | Parameter Indicator | |||
| TP-SCTS | x | M | x | 7 octets | Service Centre Time Stamp | |||
| TP-PID | M | O | M | O | O | M | 1 octet | Protocol Identifier |
| TP-DCS | M | O | M | O | O | 1 octet | Data Coding Scheme | |
| TP-SCTS | M | x | x | 7 octets | Service Centre Time Stamp | |||
| TP-VP | O | 0, 1 or 7 octets | Validity Period | |||||
| TP-UDL | M | O | M | O | O | 1 octet | User Data Length | |
| TP-UD | O | O | O | O | O | given by TP-UDL | User Data | |
| TP-CT | M | 1 octet | Command Type | |||||
| TP-MN | M | 1 octet | Message Number | |||||
| TP-DA | x | M | 2-12 octets | Destination Address | ||||
| TP-CDL | M | 1 octet | Command Data Length | |||||
| TP-CD | O | given by TP-CDL | Command Data | |||||
The first octet of the TPDU contains various flags including the TP-MTI field described above:
| bit(s) | Meaning |
|---|---|
| 1-0 | TP-Message-Type-Indicator (TP-MTI) |
| 2 | TP-More-Messages-to-Send (TP-MMS) in SMS-DELIVER (0 = more messages) |
| 2 | TP-Reject-Duplicates (TP-RD) in SMS-SUBMIT |
| 3 | TP-Loop-Prevention (TP-LP) in SMS-DELIVER and SMS-STATUS-REPORT |
| 4-3 | TP-Validity-Period-Format (TP-VPF) in SMS-SUBMIT (00 = not present) |
| 5 | TP-Status-Report-Indication (TP-SRI) in SMS-DELIVER |
| 5 | TP-Status-Report-Request (TP-SRR) in SMS-SUBMIT and SMS-COMMAND |
| 5 | TP-Status-Report-Qualifier (TP-SRQ) in SMS-STATUS-REPORT |
| 6 | TP-User-Data-Header-Indicator (TP-UDHI) |
| 7 | TP-Reply-Path (TP-RP) in SMS-DELIVER and SMS-SUBMIT |
By setting the TP-More-Messages-to-Send (TP-MMS) bit to 0 (reversed logic), the SMSC signals it has more messages for the recipient (often further segments of a concatenated message). The MSC usually does not close the connection to the mobile phone and does not end the MAP dialogue with the SMSC, which allows faster delivery of subsequent messages or message segments. If by coincidence the further messages vanish from the SMSC in the meantime (when they are for example deleted), the SMSC terminates the MAP dialogue with a MAP Abort message.
The TP-Loop-Prevention (TP-LP) bit is designed to prevent looping of SMS-DELIVER or SMS-STATUS-REPORT messages routed to a different address than is their destination address or generated by an application. Such message may be sent only if the original message had this flag cleared and the new message must be sent with the flag set.
By setting the TP-Status-Report-Indication (TP-SRI) bit to 1, the SMSC requests a status report to be returned to the SME.
By setting the TP-Status-Report-Request (TP-SRR) bit to 1 in a SMS-SUBMIT or SMS-COMMAND, the mobile phone requests a status report to be returned by the SMSC.
When the TP-SRQ has value of 1 in an SMS-STATUS-REPORT message, the message is the result of an SMS-COMMAND; otherwise it is a result of an SMS-SUBMIT.
When TP-UDHI has value 1, the TP-UD field starts with User Data Header.
Setting the TP-RP bits turns on a feature which allows to send a reply for a message using the same path as the original message. If the originator and the recipient home networks differ, the reply would go through another SMSC then usually. The mobile operator must take special measures to charge such messages.
Both SM-RP and MAP used to transmit GSM 03.40 TPDU carry enough information to return acknowledgement—the information whether a request was successful or not. However, a GSM 03.40 TPDU may be included in the acknowledgement to carry even more information. The GSM 03.40 has undergone the following development:
- Up to GSM 03.40 5.2.0 SMS-DELIVER-REPORT and SMS-SUBMIT-REPORT was sent only in the case of an error. Since 5.3.0 they are sent in case of success as well. MO-ForwardSM-Res was introduced back in GSM 09.02 5.6.0 August 1997
- Up to GSM 03.40 6.0.0 SMS-DELIVER-REPORT and SMS-SUBMIT-REPORT sent in case of an error contained only TP-MTI and TP-FCS fields and the last field in SMS-STATUS-REPORT was TP-ST. Since version 6.1.0 these TPDUs has format shown in the table above.
Although these changes are ancient (version 6.1.0 occurred in July 1998), old formats of MAP are frequently seen even in today's networks.
Message Content
The content of the message (its text when the message is not a binary one) is carried in the TP-UD field. Its size may be up to 160 × 7 = 140 × 8 = 1120 bits. Longer messages can be split to multiple parts and sent as a Concatenated SMS. The length of message content is given in the TP-UDL field. When the message encoding is GSM 7-bit default alphabet (depends on TP-DCS field), the TP-UDL gives length of TP-UD in 7-bit units; otherwise TP-UDL gives length of the TP-UD in octets.
When TP-UDHI is 1, the TP-UD starts with User Data Header (UDH); in this case the first octet of the TP-UD is User Data Header Length (UDHL) octet, containing the length of the UDH in octets without UDHL itself. UDH eats room from the TP-UD field. When the message encoding is GSM 7-bit default alphabet and a UDH is present, fill bits are inserted to align start of the first character of the text after UDH with septet boundary. This behaviour was designed for older mobile phones which don't understand UDH; such mobile phones might display the UDH as a jumble of strange characters; if the first character after UDH was Carriage Return (CR), the mobile phone would rewrite the mess with the rest of the message.
Addresses
A GSM 03.40 message contains at most one address: destination address (TP-DA) in SMS-SUBMIT and SMS-COMMAND, originator address (TP-OA) in SMS-DELIVER and recipient address (TP-RA) in SMS-STATUS-REPORT. Other addresses are carried by lower layers.
The format of addresses in the GSM 03.40 is described in the following table:
| octet | Meaning |
|---|---|
| 0 | address length in nibbles (semi-octets) |
| 1 | EXT, TON, NPI |
| 2-11 | address digits |
Type of number (TON):
| Bit 6 5 4 | Meaning |
|---|---|
| 0 0 0 | Unknown 1) |
| 0 0 1 | International number 2) |
| 0 1 0 | National number 3) |
| 0 1 1 | Network specific number 4) |
| 1 0 0 | Subscriber number 5) |
| 1 0 1 | Alphanumeric, (coded according to 3GPP TS 23.038 [9] GSM 7-bit default alphabet) |
| 1 1 0 | Abbreviated number |
| 1 1 1 | Reserved for extension |
If a subscriber enters a telephone number with `+' sign at its start, the `+' sign will be removed and the address gets TON=1 (international number), NPI=1. The number itself must always start with a country code and must be formatted exactly according to the E.164 standard.
In contrast, for numbers written without `+' sign the address gets TON=0 (unknown), NPI=1. In this case the number must adhere to the mobile operator's dial plan, which means that international numbers must have the international prefix (00 in most countries, but 011 in the USA) before the country code and numbers for long-distance calls must start with the trunk prefix (0 in most countries, 1 in the USA) followed by a trunk code.
Numbering plan identification (NPI):
| Bits 3 2 1 0 | Meaning |
|---|---|
| 0 0 0 0 | Unknown |
| 0 0 0 1 | ISDN/telephone numbering plan (E.164/E.163) |
| 0 0 1 1 | Data numbering plan (X.121) |
| 0 1 0 0 | Telex numbering plan |
| 0 1 0 1 | Service Centre Specific plan 1) |
| 0 1 1 0 | Service Centre Specific plan 1) |
| 1 0 0 0 | National numbering plan |
| 1 0 0 1 | Private numbering plan |
| 1 0 1 0 | ERMES numbering plan (ETSI DE/PS 3 01 3) |
| 1 1 1 1 | Reserved for extension |
Telephone numbers should have NPI=1. Application servers may use alphanumeric addresses which have TON=5, NPI=0 combination.
The EXT bit is always 1 meaning "no extension".
Address examples
U.S. number +1 555 123 4567 would be encoded as 0B 91 51 55 21 43 65 F7 (the F in upper four bits of the last octet is a filler which is used when the number length is odd).
Alphanumeric address is at first put to the GSM 7-bit default alphabet, then encoded the same way as any message text in TP-UD field (that means it is 7-bit packed) and then the address is supplied with the "number" length and TON and NPI.
For example, a fictional alphanumeric address Design@Home is converted to the GSM 7-bit default alphabet which yields 11 bytes 44 65 73 69 67 6E 00 48 6F 6D 65 (hex), the 7-bit packing transforms it to 77 bits stored in 10 octets as C4 F2 3C 7D 76 03 90 EF 76 19; 77 bits is 20 nibbles (14 hex) which is the value of the first octet of the address. The second octet contains TON (5) and NPI (0), which yields D0 hex. The complete address in the GSM format is 14 D0 C4 F2 3C 7D 76 03 90 EF 76 19.
Message Reference
The Message Reference field (TP-MR) is used in all messages on the submission side with exception of the SMS-SUBMIT-REPORT (that is in SMS-SUBMIT, SMS-COMMAND and SMS-STATUS-REPORT). It is a single-octet value which is incremented each time a new message is submitted or a new SMS-COMMAND is sent. If the message submission fails, the mobile phone should repeat the submission with the same TP-MR value and with the TP-RD bit set to 1.
Time Format
A date and time used in TP-SCTS, TP-DT and in Absolute format of TP-VP is stored in 7 octets:
| octet | Content |
|---|---|
| 0 | Last two digits of the year |
| 1 | Month |
| 2 | Day |
| 3 | Hour |
| 4 | Minute |
| 5 | Second |
| 6 | Time zone |
In all octets the values are stored in binary coded decimal format with switched digits (number 35 is stored as 53 hex).
Time zone is given in quarters of an hour. If the time zone offset is negative (in Western hemisphere) bit 3 of the last octet is set to 1.
23:01:56 Mar 25th 2013 PST (GMT-7) would be encoded as 31 30 52 32 10 65 8A.
In this example, the time zone, 8A is binary 1000 1010. Bit 3 is 1, therefore the time zone is negative. The remaining number (bit-wise 'and' with 1111 0111) is 1000 0010, hexadecimal 82. Treat this as any previous element in the sequence, (hex 82 represents number 28). Finally the time zone offset is given by 28 × 15 minutes = 420 minutes (7 hours).
Validity Period
An SMS-SUBMIT TPDU may contain a TP-VP parameter which limits the time period for which the SMSC would attempt to deliver the message. However, the validity period is usually limited globally by the SMSC configuration parameter— often to 48 or 72 hours. The Validity Period format is defined by the Validity Period Format field:
| TP-VPF | TP-VP format | TP-VP length |
|---|---|---|
| 0 0 | TP-VP not present | 0 |
| 0 1 | Enhanced format | 7 |
| 1 0 | Relative format | 1 |
| 1 1 | Absolute format | 7 |
Relative format
| TP-VP value | Validity period | Possible validity periods |
|---|---|---|
| 0–143 | (TP-VP + 1) x 5 minutes | 5, 10, 15 minutes ... 11:55, 12:00 hours |
| 144–167 | (12 + (TP-VP - 143) / 2 ) hours | 12:30, 13:00, ... 23:30, 24:00 hours |
| 168–196 | (TP-VP - 166) days | 2, 3, 4, ... 30 days |
| 197–255 | (TP-VP - 192) weeks | 5, 6, 7, ... 63 weeks |
Absolute format
The absolute format is identical to the other time formats in GSM 03.40.
Enhanced format
Enhanced format of TP-VP field is seldom used. It has always 7 octets, although some of them are not used. The first octet is TP-VP Functionality Indicator. Its 3 least significant bits have the following meaning:
| 2 1 0 | Meaning |
|---|---|
| 0 0 0 | No validity period specified |
| 0 0 1 | The following octet is a relative validity period as described in the Relative Validity Period Values table |
| 0 1 0 | The following octet contains a relative validity period in seconds in the range 0 to 255 |
| 0 1 1 | The following 3 octets contain a relative validity period in hours, minutes and seconds as the 3rd to 5th octet of time format |
| 1 X X | Reserved |
The value of 1 in the bit 6 of the first octet means that the message is Single-shot. The value of 1 in the bit 7 of the first octet indicates that TP-VP functionality indicator extends to another octet. However, no such extensions are defined.
Protocol Identifier
TP-PID (Protocol identifier) either refers to the higher layer protocol being used, indicates interworking with a certain type of telematic device (like fax, telex, pager, teletex, e-mail), specifies replace type of the message or allows download of configuration parameters to the SIM card. Plain MO-MT messages have PID=0.
| TP-PID | meaning |
|---|---|
| 0 | Default store and forward short message |
| 1–31 | no telematic interworking, but SME to SME protocol |
| 32 | implicit telemetic device |
| 33 | Telex or teletex reduced to telex format |
| 34 | Group 3 telefax |
| 35 | Group 4 telefax |
| 36 | Voice telephone |
| 37 | ERMES (European Radio Messaging System) |
| 38 | National Paging system (known to the SC) |
| 39 | Videotex (T.100 [20] /T.101 [21]) |
| 40 | Teletex, carrier unspecified |
| 41 | Teletex, in PSPDN |
| 42 | Teletex, in CSPDN |
| 43 | Teletex, in analog PSTN |
| 44 | Teletex, in digital ISDN |
| 45 | UCI (Universal Computer Interface, ETSI DE/PS 3 01 3) |
| 46–47 | Reserved |
| 48 | A message handling facility (known to the SC) |
| 49 | Any public X.400 based message handling system |
| 50 | Internet Electronic Mail |
| 51–55 | Reserved |
| 56–62 | SC-specific; usage based on mutual agreement between the SME and the SC |
| 63 | A GSM/UMTS mobile station. |
| 64 | Short Message Type 0 |
| 65 | Replace Short Message Type 1 |
| 66 | Replace Short Message Type 2 |
| 67 | Replace Short Message Type 3 |
| 68 | Replace Short Message Type 4 |
| 69 | Replace Short Message Type 5 |
| 70 | Replace Short Message Type 6 |
| 71 | Replace Short Message Type 7 |
| 72 | Device Triggering Short Message |
| 73–93 | Reserved |
| 94 | Enhanced Message Service (Obsolete) |
| 95 | Return Call Message |
| 96–123 | Reserved |
| 124 | ANSI-136 R-DATA |
| 125 | ME Data download |
| 126 | ME De personalization Short Message |
| 127 | (U)SIM Data download |
| 128–191 | reserved |
| 192–255 | Assigns bits 0 5 for SC specific use |
For TP-PID = 63 the SC converts the SM from the received TP Data Coding Scheme to any data coding scheme supported by that MS (e.g. the default).
Short Message Type 0 is known as a silent SMS. Any handset must be able to receive such short message irrespective of whether there is memory available in the (U)SIM or ME or not, must acknowledge receipt of the message, but must not indicate its receipt to the user and must discard its contents, so the message will not be stored in the (U)SIM or ME.
Data Coding Scheme
A special 7-bit encoding called GSM 7 bit default alphabet was designed for Short Message System in GSM. The alphabet contains the most-often used symbols from most Western-European languages (and some Greek uppercase letters). Some ASCII characters and the Euro sign did not fit into the GSM 7-bit default alphabet and must be encoded using two septets. These characters form GSM 7-bit default alphabet extension table. Support of the GSM 7-bit alphabet is mandatory for GSM handsets and network elements.[7]
Languages which use Latin script, but use characters which are not present in the GSM 7-bit default alphabet, often replace missing characters with diacritic marks with corresponding characters without diacritics, which causes not entirely satisfactory user experience, but is often accepted. For best look the 16-bit UTF-16 (in GSM called UCS-2) encoding may be used at price of reducing length of a (non segmented) message from 160 to 70 characters.
The messages in Chinese, Korean or Japanese languages must be encoded using the UTF-16 character encoding. The same was also true for other languages using non-Latin scripts like Russian, Arabic, Hebrew and various Indian languages. In 3GPP TS 23.038 8.0.0 published in 2008 a new feature, an extended National language shift table was introduced, which in the version 11.0.0 published in 2012 covers Turkish, Spanish, Portuguese, Bengali, Gujarati, Hindi, Kannada, Malayalam, Oriya, Punjabi, Tamil, Telugu and Urdu languages. The mechanism replaces GSM 7-bit default alphabet code table and/or extended table with a national table(s) according to special information elements in User Data Header. The non-segmented message using national language shift table(s) may carry up to 155 (or 153) 7-bit characters.
The Data Coding Scheme (TP-DCS) field contains primarily information about message encoding. GSM recognizes only 2 encodings for text messages and 1 encoding for binary messages:
- GSM 7-bit default alphabet (which includes using of National language shift tables as well)
- UCS-2
- 8-bit data
The TP-DCS octet has a complex syntax to allow carrying of other information; the most notable are message classes:
| Value | Message Class |
|---|---|
| 0 0 | 0 - Flash messages |
| 0 1 | 1 - ME-specific |
| 1 0 | 2 - SIM / USIM specific |
| 1 1 | 3 - TE-specific |
Flash messages are received by a mobile phone even though it has full memory. They are not stored in the phone, they just displayed on the phone display.
Another feature available through TP-DCS is Automatic Deletion: after reading the message is deleted from the phone.
Message Waiting Indication group of DCS values can set or reset flags of indicating presence of unread voicemail, fax, e-mail or other messages.
A special DCS values also allows message compression, but it perhaps is not used by any operator.
The values of TP-DCS are defined in GSM recommendation 03.38. Messages sent via this encoding can be encoded in the default GSM 7-bit alphabet, the 8-bit data alphabet, and the 16-bit UCS-2 alphabet.[7]
Discharge Time
The TP-DT field indicates the time and date associated with a particular TP-ST outcome:
- if the message has been delivered or, more generally, other transaction completed (TP-ST is 0-31), the TP-DT is the time of the completion of the transaction
- if the SMSC is still trying to deliver the message (TP-ST is 32-63), the TP-DT is the time of the last delivery attempt
- if the SMSC is not making any more delivery attempts (TP-ST is 64-127), the TP-DT is either the time of the last delivery attempt or the time at which the SMSC disposed the message
Parameter Indicator
The TP-PI field indicates presence of further fields in the SUBMIT-REPORT, DELIVER-REPORT or SMS-STATUS-REPORT TPDU.
| bit | Meaning |
|---|---|
| 0 | TP-PID |
| 1 | TP-DCS |
| 2 | TP-UDL and TP-UD |
| 8 | another TP-PI octet (extension bit) |
As currently there are still four free bits in TP-PI, it can be expected that the extension bit will be zero even in the future, which helps to distinguish TP-PI field from TP-FCS field when information whether TPDU is part of positive or negative response is not available: if the most significant bit of the second octet of TPDU is 1, the second octet is TP-FCS (in a negative response), otherwise it is TP-PI (in a positive response).
See also
References
- 3GPP TS 23.040 3rd Generation Partnership Project; Technical realization of the Short Message Service (SMS)
- 3GPP TS 24.011 3rd Generation Partnership Project; Point-to-Point Short Message Service (SMS) support on mobile radio interface
- 3GPP TS 29.002 3rd Generation Partnership Project; Mobile Application Part (MAP) specification
- 3rd Generation Partnership Project; Technical realization of the Short Message Service (SMS) (3G TS 23.040 version 11.5.0) (zipped .doc file), ETSI, March 2013.
- 3GPP TS 24.341 3rd Generation Partnership Project; Support of SMS over IP networks
- 3GPP TS 24.451 Support of SMS and MMS over NGN IMS subsystem; Stage 3 of 3GPP TS 24.341 Release 7
- 3GPP TS 23.038, Alphabets and language-specific information.