Deukhoofd
/
MalachScript
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Fixes fallthrough in lexer on initial bytes of Byte order mark (\xEF), but not the following ones.

This commit is contained in:
Deukhoofd 2020-11-01 13:42:46 +01:00
parent 329a54c8cf
commit 14db160890
Signed by: Deukhoofd
GPG Key ID: F63E044490819F6F
1 changed files with 2 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
src/Parser/Lexer/Lexer.cpp
View File

@ -250,6 +250,8 @@ namespace MalachScript::Parser {
Progress(2); Progress(2);
return Create<LexTokenImpl<LexTokenKind::Whitespace>>(TextSpan(start, start + 3)); return Create<LexTokenImpl<LexTokenKind::Whitespace>>(TextSpan(start, start + 3));
} }
LogError(Diagnostics::DiagnosticType::UnknownToken, TextSpan(start, start + 1));
return Create<LexTokenImpl<LexTokenKind::Unknown>>(TextSpan(start, start + 1));
} }
case u8'0': case u8'0':
case u8'1': case u8'1':