Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add TTCN-3 testing language lexer #1337

Merged
merged 30 commits into from Nov 9, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
e1f48aa
Creare TTCN-3 lexer files
YannGarcia Sep 30, 2019
968f9a3
Add sample, Enhance TTCN-3 lexer
YannGarcia Oct 1, 2019
00c0da1
Add visual sample
YannGarcia Oct 1, 2019
6ce8b5a
Reduce demo in lines of code
pyrmont Oct 6, 2019
d646b4b
Capitalise name of lexer
pyrmont Oct 6, 2019
e2f3a23
Shorten description
pyrmont Oct 6, 2019
88df300
Wrap lists of defined words
pyrmont Oct 6, 2019
1ac95c6
Memoise defined words
pyrmont Oct 6, 2019
cd2d02f
Reorder rules to use memoised words
pyrmont Oct 7, 2019
26c2936
Add token for reserved words
pyrmont Oct 7, 2019
621906b
Rationalise operators and punctuation
pyrmont Oct 7, 2019
c11329f
Remove unnecessary newline rule
pyrmont Oct 7, 2019
55c7393
Remove duplicate characters from rules
pyrmont Oct 7, 2019
d199ed0
Tidy up comment rules
pyrmont Oct 7, 2019
6dd49f6
Use numeric metacharacters
pyrmont Oct 7, 2019
6482586
Add missing keywords & reserved identifiers
YannGarcia Oct 15, 2019
b96f243
Add rule for newline
pyrmont Oct 15, 2019
369bae6
Properly memoise defined words
pyrmont Oct 15, 2019
81c813e
Remove ws statement and useless comment
YannGarcia Oct 15, 2019
e7f9100
Replace [a-zA-Z0-9_] by [\w]
YannGarcia Oct 15, 2019
d10a945
Full review of TTCN-3 rules
YannGarcia Oct 25, 2019
2f36893
Use \w character class
pyrmont Oct 26, 2019
3a258df
Remove unnecessary builtin variable
pyrmont Oct 26, 2019
105fddd
Consolidate whitespace rules
pyrmont Oct 26, 2019
0c7313e
Insert spacing to improve visual dilenation
pyrmont Oct 26, 2019
c793060
Remove function state
pyrmont Oct 26, 2019
4bd8c85
Remove function signature rule
pyrmont Oct 26, 2019
c663f8d
Remove unnecessary line break
pyrmont Oct 26, 2019
27a7c78
Remove case state
pyrmont Oct 26, 2019
58667e5
Remove Error rule, enhance Integer rule
YannGarcia Nov 3, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 6 additions & 0 deletions lib/rouge/demos/ttcn3
@@ -0,0 +1,6 @@
module DNSTester {
pyrmont marked this conversation as resolved.
Show resolved Hide resolved
type integer Identification( 0..65535 ); // 16-bit integer
type enumerated MessageKind {e_Question, e_Answer};
type charstring Question;
type charstring Answer;
}
119 changes: 119 additions & 0 deletions lib/rouge/lexers/ttcn3.rb
@@ -0,0 +1,119 @@
# -*- coding: utf-8 -*- #
# frozen_string_literal: true

module Rouge
module Lexers
class TTCN3 < RegexLexer
pyrmont marked this conversation as resolved.
Show resolved Hide resolved
title "TTCN3"
desc "The TTCN3 programming language (ttcn-3.org)"
pyrmont marked this conversation as resolved.
Show resolved Hide resolved

tag 'ttcn3'
filenames '*.ttcn', '*.ttcn3'
mimetypes 'text/x-ttcn3', 'text/x-ttcn'

def self.keywords
pyrmont marked this conversation as resolved.
Show resolved Hide resolved
@keywords ||= %w(
module import group type port component signature external
execute const template function altstep testcase var timer if
else select case for while do label goto start stop return
break int2char int2unichar int2bit int2enum int2hex int2oct
int2str int2float float2int char2int char2oct unichar2int
unichar2oct bit2int bit2hex bit2oct bit2str hex2int hex2bit
hex2oct hex2str oct2int oct2bit oct2hex oct2str oct2char
oct2unichar str2int str2hex str2oct str2float enum2int
any2unistr lengthof sizeof ispresent ischosen isvalue isbound
istemplatekind regexp substr replace encvalue decvalue
encvalue_unichar decvalue_unichar encvalue_o decvalue_o
get_stringencoding remove_bom rnd hostid send receive
setverdict
)
end

def self.reserved
@reserved ||= %w(
all alt apply assert at configuration conjunct const control
delta deterministic disjunct duration fail finished fuzzy from
history implies inconc inv lazy mod mode notinv now omit
onentry onexit par pass prev realtime seq setstate static
stepsize stream timestamp until values wait
)
end

def self.types
@types ||= %w(
anytype address boolean bitstring charstring hexstring octetstring
component enumerated float integer port record set of union universal
)
end

# optional comment or whitespace
ws = %r((?:\s|//.*?\n|/[*].*?[*]/)+)
YannGarcia marked this conversation as resolved.
Show resolved Hide resolved
id = /[a-zA-Z_]\w*/
digit = /\d_+\d|\d/
bin_digit = /[01]_+[01]|[01]/
oct_digit = /[0-7]_+[0-7]|[0-7]/
hex_digit = /\h_+\h|\h/

state :statements do
rule %r/\n+/m, Text
rule %r/[ \t\r]+/, Text
rule %r/\\\n/, Text # line continuation

rule %r(//(\\.|.)*?$), Comment::Single
rule %r(/(\\\n)?[*].*?[*](\\\n)?/)m, Comment::Multiline

rule %r/"/, Str, :string
rule %r/'(?:\\.|[^\\]|\\u[0-9a-f]{4})'/, Str::Char

rule %r/#{digit}+\.#{digit}+([eE]#{digit}+)?[fd]?/i, Num::Float
rule %r/'#{bin_digit}+'B/i, Num::Bin
rule %r/'#{hex_digit}+'H/i, Num::Hex
rule %r/'#{oct_digit}+'O/i, Num::Oct
rule %r/#{digit}+/i, Num::Integer

rule %r([~!%^&*+:=\|?<>/-]), Operator
rule %r/[()\[\]{},.;:]/, Punctuation

rule %r/(?:true|false|null)\b/, Name::Builtin

rule id do |m|
pyrmont marked this conversation as resolved.
Show resolved Hide resolved
name = m[0]
if self.class.keywords.include? name
token Keyword
elsif self.class.types.include? name
token Keyword::Type
elsif self.class.reserved.include? name
token Keyword::Reserved
else
token Name
end
end
end

state :root do
rule %r/module\b/, Keyword::Declaration, :module
rule %r/import\b/, Keyword::Namespace, :import

mixin :statements
end

state :string do
rule %r/"/, Str, :pop!
rule %r/\\([\\abfnrtv"']|x[a-fA-F0-9]{2,4}|[0-7]{1,3})/, Str::Escape
rule %r/[^\\"\n]+/, Str
rule %r/\\\n/, Str
rule %r/\\/, Str # stray backslash
end

state :module do
rule %r/\s+/m, Text
rule id, Name::Class, :pop!
end

state :import do
rule %r/\s+/m, Text
rule %r/[\w.]+\*?/, Name::Namespace, :pop!
end
end
end
end
20 changes: 20 additions & 0 deletions spec/lexers/ttcn3_spec.rb
@@ -0,0 +1,20 @@
# -*- coding: utf-8 -*- #
# frozen_string_literal: true

describe Rouge::Lexers::TTCN3 do
let(:subject) { Rouge::Lexers::TTCN3.new }

describe 'guessing' do
include Support::Guessing

it 'guesses by filename' do
assert_guess :filename => 'foo.ttcn'
assert_guess :filename => 'foo.ttcn3'
end

it 'guesses by mimetype' do
assert_guess :mimetype => 'text/x-ttcn'
assert_guess :mimetype => 'text/x-ttcn3'
end
end
end
120 changes: 120 additions & 0 deletions spec/visual/samples/ttcn3
@@ -0,0 +1,120 @@
/* ----------------------------------------------------------------------------
* (c) Copyright Wiley & Sons 2005
*
* @author: Colin Willcock, Thomas Deiß, Stephan Tobies, Stefan Keil, Federico
* Engler, Stephan Schulz
* @desc: This is a strongly simplified Domain Name Server (DNS) test suite
* for testing some basic domain name resolution behaviour.
* @remark: This TTCN-3 code is based on the DNS example code presented in
* "C. Willock et al., An Introduction to TTCN-3, Wiley & Sons, 2005.
* ISBN: 0-470-01224-2"
* This copyright notice shall not be removed in copies of this file.
* ----------------------------------------------------------------------------
*/
module DNSTester {

// Simple type definitions to match the protocol structure
type integer Identification( 0..65535 ); // 16-bit integer
type enumerated MessageKind {e_Question, e_Answer};
type charstring Question;
type charstring Answer;

// The definition of our DNS message type.
type record DNSMessage {
Identification identification,
MessageKind messageKind,
Question question,
Answer answer optional
}

// A possible template for the DNS message type.
template DNSMessage a_NokiaQuestion := {
identification := 12345,
messageKind := e_Question,
question := "www.nokia.com",
answer := omit
}

// A parameterized template for DNS questions based on DNSMessage.
template DNSMessage a_DNSQuestion( Identification p_id, Question p_question ) := {
identification := p_id,
messageKind := e_Question,
question := p_question,
answer := omit
}

// A parameterized template for DNS answers based on DNSMessage.
template DNSMessage a_DNSAnswer( Identification p_id, Answer p_answer ) := {
identification := p_id,
messageKind := e_Answer,
question := ?,
answer := p_answer
}

// DNS messages are allowed to move in and out through ports of this type.
type port DNSPort message {
inout DNSMessage
}

// Our single component uses one single port to communicate with the SUT.
type component DNSClient {
port DNSPort serverPort
}

// Our first test case! This small test case will behave very poorly in case
// of an erroneous SUT. More about this later!
testcase ExampleResolveNokia1() runs on DNSClient {
serverPort.send( a_DNSQuestion( 12345, "www.research.nokia.com" ) );
serverPort.receive( a_DNSAnswer( 12345, "172.21.56.98" ) );
setverdict( pass );
stop;
}

testcase ExampleResolveNokia2() runs on DNSClient {
serverPort.send( a_DNSQuestion( 12345, "www.research.nokia.com" ) );
alt {
// Handle the case when the expected answer comes in.
[] serverPort.receive( a_DNSAnswer( 12345, "172.21.56.98" ) ) {
setverdict( pass );
}
// Handle the case when unexpected answers come in.
[] serverPort.receive {
setverdict( fail );
}
}
stop;
}

// Our test case is now able to handle incorrect replies as well as
// missing replies.
testcase ExampleResolveNokia3() runs on DNSClient {
timer replyTimer;
serverPort.send( a_DNSQuestion( 12345, "www.research.nokia.com" ) );
replyTimer.start( 20.0 );
alt {
// Handle the case when the expected answer comes in.
[] serverPort.receive( a_DNSAnswer( 12345, "172.21.56.98" ) ) {
setverdict( pass );
replyTimer.stop;
}
// Handle the case when unexpected answers come in.
[] serverPort.receive {
setverdict( fail );
replyTimer.stop;
}
// Handle the case when no answer comes in.
[] replyTimer.timeout {
setverdict( fail );
}
}
stop;
}

// Our small control part.
control {
execute( ExampleResolveNokia1() );
execute( ExampleResolveNokia2() );
execute( ExampleResolveNokia3() );
}

}