Preparation, Enforcement, and Comparison of Internationalized Strings (PRECIS) Test Vectors
Atlanta
GA
USA
sam@samwhited.com
https://blog.samwhited.com/
Applications
PRECIS
tests
unicode
This document contains test vectors for several Preparation,
Enforcement, and Comparison of Internationalized Strings (PRECIS)
profiles.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task
Force (IETF). Note that other groups may also distribute working
documents as Internet-Drafts. The list of current Internet-Drafts is
at .
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on 27 October 2020.
Copyright Notice
Copyright (c) 2020 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
() in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with
respect to this document. Code Components extracted from this
document must include Simplified BSD License text as described in
Section 4.e of the Trust Legal Provisions and are provided without
warranty as described in the Simplified BSD License.
Table of Contents
-
. Introduction
-
. Test Vectors
-
. Methodology
-
. Enforcement Tests
-
. Basic
-
. Context Rule 1
-
. Context Rule 2
-
. Context Rule 3
-
. Context Rule 4
-
. Context Rule 5+6
-
. Context Rule 7
-
. Context Rule 8+9
-
. Nickname
-
. OpaqueString
-
. UsernameCaseMapped
-
. UsernameCasePreserved
-
. UsernameCaseMappedRestricted
-
. Comparison Tests
-
. IANA Considerations
-
. Security Considerations
-
. References
-
. Normative References
-
. Informative References
-
. Acknowledgements
-
Author's Address
Introduction
Overview
This document contains test vectors for several Preparation,
Enforcement, and Comparison of Internationalized Strings (PRECIS) profiles.
Two sets of vectors exist for profiles of the base classes.
The first set tests the enforcement step.
It takes the form of a table that contains an input string, an output
string and an optional error.
If the error column has a value of "ctx" it indicates that the context
rule was violated, "empty" means that an empty string was disallowed,
"rune" means that a disallowed code point (rune) was encountered, and
"bidi" indicates that the BIDI Rule was violated.
The second set tests the PRECIS comparison step.
It takes the form of a table that contains two strings, a boolean that
is TRUE if the two strings should be considered equal and FALSE
otherwise, and an optional comment.
Vectors for inclusion of code points in the IdentifierClass and
FreeformClass are not included.
A complete list of valid inputs and Outputs for the derived property
algorithm specified in can be
found in the IANA
"PRECIS Derived Property Value" registry.
Terminology
Many important terms used in this document are defined in
, , and
.
Test Vectors
Methodology
The test vectors in this document have been verified by three
independent implementations.
An implementation that concurs with the results provided in this
document should be interoperable with other implementations.
Input and output strings are expressed as UTF-8 and hexidecimal encoded
byte strings.
Enforcement Tests
Basic
| Input |
Output |
Err |
| 65cc81cc9f |
c3a9cc9f |
|
Context Rule 1
| Input |
Output |
Err |
| e2808c |
|
ctx |
| e2808c61 |
|
ctx |
| 61e2808c |
|
ctx |
| e2808cd8a7 |
|
ctx |
| d8ace2808c |
|
ctx |
| d890e2808cd890d8a7 |
|
ctx |
| d8acd890e2808cd890 |
|
ctx |
| d8ace2808cd8a7 |
d8ace2808cd8a7 |
|
| d8acd890e2808cd890d8a7 |
d8acd890e2808cd890d8a7 |
|
| d8acd890d890e2808cd890d890d8a7 |
d8acd890d890e2808cd890d890d8a7 |
|
| d8acd890e2808cd8a7 |
d8acd890e2808cd8a7 |
|
| d8ace2808cd890d8a7 |
d8ace2808cd890d8a7 |
|
| eaa1b2e2808cd8ac |
eaa1b2e2808cd8ac |
|
| eaa1b2d890e2808cd890d8ac |
eaa1b2d890e2808cd890d8ac |
|
| eaa1b2d890d890e2808cd890d890d8ac |
eaa1b2d890d890e2808cd890d890d8ac |
|
| eaa1b2d890e2808cd8ac |
eaa1b2d890e2808cd8ac |
|
| eaa1b2e2808cd890d8ac |
eaa1b2e2808cd890d8ac |
|
| e0a98de2808c |
e0a98de2808c |
|
| eaa1b2e0a98de2808c |
eaa1b2e0a98de2808c |
|
| eaa1b2e0a98dd890e2808c |
|
ctx |
| eaa1b2e0a98dd890e2808c |
|
ctx |
| e0ab8de2808c |
e0ab8de2808c |
|
| eaa1b2e0ab8de2808c |
eaa1b2e0ab8de2808c |
|
| eaa1b2e0ab8dd890e2808c |
|
ctx |
| eaa1b2e0ab8dd890e2808c |
|
ctx |
| eaa1b2e0ab8de2808cd8ac |
eaa1b2e0ab8de2808cd8ac |
|
| eaa1b2e2808ce0ab8dd8ac |
eaa1b2e2808ce0ab8dd8ac |
|
Context Rule 2
| Input |
Output |
Err |
| e2808d |
|
ctx |
| e2808d61 |
|
ctx |
| 61e2808d |
|
ctx |
| e0a98de2808d |
e0a98de2808d |
|
| eaa1b2e0a98de2808d |
eaa1b2e0a98de2808d |
|
| e0a98d61e2808d |
|
ctx |
Context Rule 3
| Input |
Output |
Err |
| c2b7 |
|
ctx |
| 6cc2b7 |
|
ctx |
| c2b76c |
|
ctx |
| 61c2b7 |
|
ctx |
| 6cc2b761 |
|
ctx |
| 61c2b761 |
|
ctx |
| 6cc2b76c |
6cc2b76c |
|
| 616cc2b76c61 |
616cc2b76c61 |
|
Context Rule 4
| Input |
Output |
Err |
| cdb5 |
|
ctx |
| cdb561 |
|
ctx |
| ceb1cdb5 |
|
ctx |
| cdb5ceb1 |
cdb5ceb1 |
|
| ceb1cdb5ceb1 |
ceb1cdb5ceb1 |
|
| cdb5cdb5ceb1 |
cdb5cdb5ceb1 |
|
| ceb1cdb5cdb5ceb1 |
ceb1cdb5cdb5ceb1 |
|
| ceb1cdb5cdb5 |
|
ctx |
| ceb1cdb5cdb561 |
|
ctx |
Context Rule 5+6
| Input |
Output |
Err |
| d7b3 |
|
ctx |
| d7b3d794 |
|
ctx |
| 61d7b362 |
|
ctx |
| d7a9d7b3 |
d7a9d7b3 |
|
| d7a9d7b3d7b3d7b3 |
d7a9d7b3d7b3d7b3 |
|
| d7b4 |
|
ctx |
| d7b4d794 |
|
ctx |
| 61d7b462 |
|
ctx |
| d7a9d7b4 |
d7a9d7b4 |
|
| d7a9d7b4d7b4d7b4 |
d7a9d7b4d7b4d7b4 |
|
| 61d7a9d7b4d7b4d7b4 |
61d7a9d7b4d7b4d7b4 |
|
Context Rule 7
| Input |
Output |
Err |
| e383bb |
|
ctx |
| 616263e383bb |
|
ctx |
| e383bb646566 |
|
ctx |
| 616263e383bb646566 |
|
ctx |
| 61e3838563e383bb646566 |
61e3838563e383bb646566 |
|
| 616263e383bb64e381b666 |
616263e383bb64e381b666 |
|
| e2ba906263e383bb646566 |
e2ba906263e383bb646566 |
|
Context Rule 8+9
| Input |
Output |
Err |
| d9a1d9a2d9a3d9a4d9a5dbb6 |
|
ctx |
| dbb1dbb2dbb3dbb4dbb5d9a6 |
|
ctx |
| d9a1d9a2d9a3d9a4d9a5 |
d9a1d9a2d9a3d9a4d9a5 |
|
| dbb1dbb2dbb3dbb4dbb5 |
dbb1dbb2dbb3dbb4dbb5 |
|
Nickname
| Input |
Output |
Err |
| 20205377616e20206f6620202041766f6e202020 |
5377616e206f662041766f6e |
|
|
|
empty |
| 20 |
|
empty |
| 2020 |
|
empty |
| 61c2a061e19a8061e2808061e2808161e2808261e2808361e2808461e2808561e2808661e2808761e2808861e2808961e2808a61e280af61e2819f61e3808061 |
612061206120612061206120612061206120612061206120612061206120612061 |
|
| 466f6f |
466f6f |
|
| 666f6f |
666f6f |
|
| 466f6f20426172 |
466f6f20426172 |
|
| 666f6f20626172 |
666f6f20626172 |
|
| cea3 |
cea3 |
|
| cf83 |
cf83 |
|
| cf82 |
cf82 |
|
| e2999a |
e2999a |
|
| 5269636861726420e285a3 |
52696368617264204956 |
|
| e284ab |
c385 |
|
| efac80 |
6666 |
|
| d7a961 |
d7a961 |
|
| eb8f99ec9dbceca1b0eab1b4ebb380eab2bded9788eb9dbd |
eb8f99ec9dbceca1b0eab1b4ebb380eab2bded9788eb9dbd |
|
OpaqueString
| Input |
Output |
Err |
| 20205377616e20206f6620202041766f6e202020 |
20205377616e20206f6620202041766f6e202020 |
|
|
|
empty |
| 20 |
20 |
|
| 2020 |
2020 |
|
| 61c2a061e19a8061e2808061e2808161e2808261e2808361e2808461e2808561e2808661e2808761e2808861e2808961e2808a61e280af61e2819f61e3808061 |
612061206120612061206120612061206120612061206120612061206120612061 |
|
| 466f6f |
466f6f |
|
| 666f6f |
666f6f |
|
| 466f6f20426172 |
466f6f20426172 |
|
| 666f6f20626172 |
666f6f20626172 |
|
| cf83 |
cf83 |
|
| 5269636861726420e285a3 |
5269636861726420e285a3 |
|
| e284ab |
c385 |
|
| 4a61636b206f6620e299a673 |
4a61636b206f6620e299a673 |
|
| 6d7920636174206973206120096279 |
|
rune |
| d7a961 |
d7a961 |
|
UsernameCaseMapped
| Input |
Output |
Err |
| 6a756c696574406578616d706c652e636f6d |
6a756c696574406578616d706c652e636f6d |
|
| 6675737362616c6c |
6675737362616c6c |
|
| 6675c39f62616c6c |
6675c39f62616c6c |
|
| cf80 |
cf80 |
|
| cea3 |
cf83 |
|
| cf83 |
cf83 |
|
| cf82 |
cf82 |
|
| 49 |
69 |
|
| 49 |
69 |
|
| cf92 |
|
rune |
| ceb0 |
ceb0 |
|
| 666f6f20626172 |
|
rune |
| e2999a |
|
bidirule: failed Bidi Rule |
| 7e |
7e |
|
| 61 |
61 |
|
| 21 |
21 |
|
| c2b2 |
|
bidirule: failed Bidi Rule |
| 09 |
|
rune |
| 0a |
|
rune |
| e29b96 |
|
bidirule: failed Bidi Rule |
| e29bbf |
|
bidirule: failed Bidi Rule |
| efac80 |
|
rune |
| e19a80 |
|
bidirule: failed Bidi Rule |
| 20 |
|
rune |
| 2020 |
|
rune |
| c785 |
|
rune |
| e19bae |
|
rune |
| d288 |
|
bidirule: failed Bidi Rule |
| e284ab |
c3a5 |
|
| 41cc8a |
c3a5 |
|
| c385 |
c3a5 |
|
| c3a7 |
c3a7 |
|
| 63cca7 |
c3a7 |
|
| c598 |
c599 |
|
| 52cc8c |
c599 |
|
| e1b9a1 |
e1b9a1 |
|
| d090 |
d0b0 |
|
| efbca1efbca2 |
6162 |
|
| d7a963 |
|
bidirule: failed Bidi Rule |
UsernameCasePreserved
| Input |
Output |
Err |
| 414243 |
414243 |
|
| efbca1efbca2 |
4142 |
|
| d7a963 |
|
bidirule: failed Bidi Rule |
| efac80 |
|
rune |
| e284ab |
c385 |
|
| e1ba9b |
|
rune |
UsernameCaseMappedRestricted
| Input |
Output |
Err |
| 6a756c696574406578616d706c652e636f6d |
|
rune |
| 49 |
69 |
|
Comparison Tests
Nickname
| A |
B |
Result |
| 61 |
62 |
false |
| 20205377616e20206f6620202041766f6e202020 |
7377616e206f662061766f6e |
true |
| 466f6f |
666f6f |
true |
| 666f6f |
666f6f |
true |
| 466f6f20426172 |
666f6f20626172 |
true |
| 666f6f20626172 |
666f6f20626172 |
true |
| cea3 |
cf83 |
true |
| cea3 |
cf82 |
false |
| cf83 |
cf82 |
false |
| 5269636861726420e285a3 |
72696368617264206976 |
true |
| e284ab |
c3a5 |
true |
| efac80 |
6666 |
true |
| c39f |
7353 |
false |
| c2a8 |
20cc88 |
true |
| c2a8 |
cc88 |
true |
| 20cc88 |
cc88 |
true |
IANA Considerations
This document requires no interaction with the Internet Assigned Numbers
Authority (IANA).
Security Considerations
This document is intended to provide test vectors for the PRECIS framework
and its profiles.
No assertion of the security of the PRECIS framework or its profiles for
any particular use is intended.
The reader is referred to for a
discussion of the general security of the PRECIS framework.
References
Normative References
PRECIS Framework: Preparation, Enforcement, and Comparison of Internationalized Strings in Application Protocols
Application protocols using Unicode code points in protocol strings need to properly handle such strings in order to enforce internationalization rules for strings placed in various protocol slots (such as addresses and identifiers) and to perform valid comparison operations (e.g., for purposes of authentication or authorization). This document defines a framework enabling application protocols to perform the preparation, enforcement, and comparison of internationalized strings ("PRECIS") in a way that depends on the properties of Unicode code points and thus is more agile with respect to versions of Unicode. As a result, this framework provides a more sustainable approach to the handling of internationalized strings than the previous framework, known as Stringprep (RFC 3454). This document obsoletes RFC 7564.
Informative References
PRECIS Derived Property Value
Terminology Used in Internationalization in the IETF
This document provides a list of terms used in the IETF when discussing internationalization. The purpose is to help frame discussions of internationalization in the various areas of the IETF and to help introduce the main concepts to IETF participants. This memo documents an Internet Best Current Practice.
The Unicode Standard
The Unicode Consortium
Acknowledgements
The author would like to thank Marcel van Lohuizen of Google for his help
and code reviews on the Go programming language implementation of PRECIS
for which these tests were designed and for contributing many of the tests
himself.
Also, Peter Saint-Andre for his encouragement on this document.
Author's Address
Atlanta
GA
USA
sam@samwhited.com
https://blog.samwhited.com/