NSRegularExpression.firstMatchInString() fails on Linux

Mamatha_Busi · March 2, 2016, 11:57am

Hello

The following test case, that currently fails on Linux, is extracted from
some of the excluded tests inside TestFoundation/TestNSRegularExpression.

import Foundation

let searchStr = "123"
let testRegex = try NSRegularExpression.init(pattern:
"a(b|c|d)(x|y|z)*|123", options: )
let firstMatch = testRegex.firstMatchInString(searchStr, options: ,
range: NSMakeRange(0,3))

if NSEqualRanges((firstMatch!.rangeAtIndex(1), NSMakeRange(NSNotFound,0))
{
    print("Test passed")
} else {
    print("First match first capture range =
\(NSStringFromRange(firstMatch!.rangeAtIndex(1)))")
    print("Test failed")
}

In the above scenario, no capture group participates in the match. So,
firstMatch.rangeAtIndex(1) should ideally return range {NSNotFound, 0} but
the actual returned value is range {-1, 0}. Hence the failure.

Debugging on the CoreFoundation, it is observed that the value -1 is
coming from 'kCFNotFound' which is defined as -1. On the Foundation side,
'NSNotFound' is defined with the value of ‘Int.max’ (which must be
Int32.max and Int64.max on 32 and 64 bit platforms respectively).

There are a couple of queries that I have:

-> Is 'NSNotFound' in Foundation mapped to 'kCFNotFound' in
CoreFoundation?

-> If so, then why do 'NSNotFound' and 'kCFNotFound' have different values
on the Foundation and CoreFoundation respectively?

Also, I tested by modifying the 'kCFNotFound' value in
CoreFoundation/Base.subproj/CFBase.h (See below code snippet):

#if TARGET_RT_64_BIT
static const CFIndex kCFNotFound = LONG_MAX;
#else
static const CFIndex kCFNotFound = INT_MAX;
#endif

With this change the above mentioned test-case passes but a lot of other
TestFoundation tests seem to fail.

Please provide your comments on the appropriate change that is needed
here.

Thank you.

Regards
Mamatha

Mamatha Busi

Java L3 Support

IBM Software Group

India Software Labs

e-mail:
mamabusi@in.ibm.com

Java Technology India Group

Tony_Parker · March 3, 2016, 6:22pm

Hi Mamatha,

Hello

The following test case, that currently fails on Linux, is extracted from some of the excluded tests inside TestFoundation/TestNSRegularExpression.

import Foundation

let searchStr = "123"
let testRegex = try NSRegularExpression.init(pattern: "a(b|c|d)(x|y|z)*|123", options: )
let firstMatch = testRegex.firstMatchInString(searchStr, options: , range: NSMakeRange(0,3))

if NSEqualRanges((firstMatch!.rangeAtIndex(1), NSMakeRange(NSNotFound,0)) {
    print("Test passed")
} else {
    print("First match first capture range = \(NSStringFromRange(firstMatch!.rangeAtIndex(1)))")
    print("Test failed")
}

In the above scenario, no capture group participates in the match. So, firstMatch.rangeAtIndex(1) should ideally return range {NSNotFound, 0} but the actual returned value is range {-1, 0}. Hence the failure.

Debugging on the CoreFoundation, it is observed that the value -1 is coming from 'kCFNotFound' which is defined as -1. On the Foundation side, 'NSNotFound' is defined with the value of ‘Int.max’ (which must be Int32.max and Int64.max on 32 and 64 bit platforms respectively).

There are a couple of queries that I have:

-> Is 'NSNotFound' in Foundation mapped to 'kCFNotFound' in CoreFoundation?

-> If so, then why do 'NSNotFound' and 'kCFNotFound' have different values on the Foundation and CoreFoundation respectively?

They don’t have the same value; we translate it manually in most places.

Also, I tested by modifying the 'kCFNotFound' value in CoreFoundation/Base.subproj/CFBase.h (See below code snippet):

if TARGET_RT_64_BIT
static const CFIndex kCFNotFound = LONG_MAX;
#else
static const CFIndex kCFNotFound = INT_MAX;
#endif

With this change the above mentioned test-case passes but a lot of other TestFoundation tests seem to fail.

Please provide your comments on the appropriate change that is needed here.

Thank you.

I think we probably just need to check the result from the CF function and translate it into an NSNotFound in this one particular case.

Thanks,
- Tony

···

On Mar 2, 2016, at 3:57 AM, Mamatha Busi via swift-corelibs-dev <swift-corelibs-dev@swift.org> wrote:

Regards
Mamatha
Mamatha Busi
<Mail Attachment.gif>
Java L3 Support
IBM Software Group
India Software Labs

e-mail: mamabusi@in.ibm.com
Java Technology India Group <http://bit.ly/a7KpQj>

_______________________________________________
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

Philippe_Hausler · March 3, 2016, 6:32pm

Hi Mamatha,

Hello

The following test case, that currently fails on Linux, is extracted from some of the excluded tests inside TestFoundation/TestNSRegularExpression.

import Foundation

let searchStr = "123"
let testRegex = try NSRegularExpression.init(pattern: "a(b|c|d)(x|y|z)*|123", options: )
let firstMatch = testRegex.firstMatchInString(searchStr, options: , range: NSMakeRange(0,3))

if NSEqualRanges((firstMatch!.rangeAtIndex(1), NSMakeRange(NSNotFound,0)) {
    print("Test passed")
} else {
    print("First match first capture range = \(NSStringFromRange(firstMatch!.rangeAtIndex(1)))")
    print("Test failed")
}

In the above scenario, no capture group participates in the match. So, firstMatch.rangeAtIndex(1) should ideally return range {NSNotFound, 0} but the actual returned value is range {-1, 0}. Hence the failure.

Debugging on the CoreFoundation, it is observed that the value -1 is coming from 'kCFNotFound' which is defined as -1. On the Foundation side, 'NSNotFound' is defined with the value of ‘Int.max’ (which must be Int32.max and Int64.max on 32 and 64 bit platforms respectively).

There are a couple of queries that I have:

-> Is 'NSNotFound' in Foundation mapped to 'kCFNotFound' in CoreFoundation?

-> If so, then why do 'NSNotFound' and 'kCFNotFound' have different values on the Foundation and CoreFoundation respectively?

They don’t have the same value; we translate it manually in most places.

Also, I tested by modifying the 'kCFNotFound' value in CoreFoundation/Base.subproj/CFBase.h (See below code snippet):

if TARGET_RT_64_BIT
static const CFIndex kCFNotFound = LONG_MAX;
#else
static const CFIndex kCFNotFound = INT_MAX;
#endif

With this change the above mentioned test-case passes but a lot of other TestFoundation tests seem to fail.

Please provide your comments on the appropriate change that is needed here.

Thank you.

I think we probably just need to check the result from the CF function and translate it into an NSNotFound in this one particular case.

I think the problem in this case is that we fetch a buffer of CFRanges and pass that to the result creation.

ranges: UnsafeMutablePointer<CFRange>
...
let result = NSTextCheckingResult.regularExpressionCheckingResultWithRanges(NSRangePointer(ranges), count: count, regularExpression: matcher.regex)

···

On Mar 3, 2016, at 10:22 AM, Tony Parker via swift-corelibs-dev <swift-corelibs-dev@swift.org> wrote:

On Mar 2, 2016, at 3:57 AM, Mamatha Busi via swift-corelibs-dev <swift-corelibs-dev@swift.org <mailto:swift-corelibs-dev@swift.org>> wrote:

Thanks,
- Tony

Regards
Mamatha
Mamatha Busi
<Mail Attachment.gif>
Java L3 Support
IBM Software Group
India Software Labs

e-mail: mamabusi@in.ibm.com <mailto:mamabusi@in.ibm.com>
Java Technology India Group <http://bit.ly/a7KpQj>

_______________________________________________
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org <mailto:swift-corelibs-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

_______________________________________________
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org <mailto:swift-corelibs-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

Tony_Parker · March 8, 2016, 5:40pm

Hi Mamatha,

@Philippe Hausler, @Tony Parker: Thank you for the response.

As per Tony's suggestions I have translated CF's kCFNotFound into an NSNotFound of Foundation in 'NSTextCheckingResult' (the place where the ranges are being constructed). Below is the code snippet with the changes:

_regularExpression = regularExpression
        super.init()
        let notFound = NSRange(location: NSNotFound,length: 0)
        for i in 0..<count {
            ranges[i].location == kCFNotFound ? _ranges.append(notFound) : _ranges.append(ranges[i])
        }

@Philippe: Request your comments on the above code changes.

TestFoundation has been run and I see no failures with the above changes.

The tests 'test_complexRegularExpressions' in the TestNSRegularExpression were excluded till date. I am now running the excluded tests as well and there are a few test-cases here which deal with the ranges having NSNotFound. Along with these should there be more tests written to test the same?

Sure, more tests are always a bonus.

Please open a PR on swift-corelibs-foundatation when you’re ready to get a review.

Thanks for tackling this!
- Tony

···

On Mar 8, 2016, at 4:15 AM, Mamatha Busi <mamabusi@in.ibm.com> wrote:

Regards
Mamatha
Mamatha Busi
<Mail Attachment.gif>
Java L3 Support
IBM Software Group
India Software Labs

e-mail: mamabusi@in.ibm.com
Java Technology India Group <http://bit.ly/a7KpQj>

From: Philippe Hausler <phausler@apple.com>
To: Tony Parker <anthony.parker@apple.com>
Cc: Mamatha Busi/India/IBM@IBMIN, swift-corelibs-dev@swift.org
Date: 03/04/2016 12:02 AM
Subject: Re: [swift-corelibs-dev] NSRegularExpression.firstMatchInString() fails on Linux
Sent by: phausler@apple.com

On Mar 3, 2016, at 10:22 AM, Tony Parker via swift-corelibs-dev <swift-corelibs-dev@swift.org <mailto:swift-corelibs-dev@swift.org>> wrote:

Hi Mamatha,

On Mar 2, 2016, at 3:57 AM, Mamatha Busi via swift-corelibs-dev <swift-corelibs-dev@swift.org <mailto:swift-corelibs-dev@swift.org>> wrote:

Hello

The following test case, that currently fails on Linux, is extracted from some of the excluded tests inside TestFoundation/TestNSRegularExpression.

import Foundation

let searchStr = "123"
let testRegex = try NSRegularExpression.init(pattern: "a(b|c|d)(x|y|z)*|123", options: )
let firstMatch = testRegex.firstMatchInString(searchStr, options: , range: NSMakeRange(0,3))

if NSEqualRanges((firstMatch!.rangeAtIndex(1), NSMakeRange(NSNotFound,0)) {
   print("Test passed")
} else {
   print("First match first capture range = \(NSStringFromRange(firstMatch!.rangeAtIndex(1)))")
   print("Test failed")
}

In the above scenario, no capture group participates in the match. So, firstMatch.rangeAtIndex(1) should ideally return range {NSNotFound, 0} but the actual returned value is range {-1, 0}. Hence the failure.

Debugging on the CoreFoundation, it is observed that the value -1 is coming from 'kCFNotFound' which is defined as -1. On the Foundation side, 'NSNotFound' is defined with the value of ‘Int.max’ (which must be Int32.max and Int64.max on 32 and 64 bit platforms respectively).

There are a couple of queries that I have:

-> Is 'NSNotFound' in Foundation mapped to 'kCFNotFound' in CoreFoundation?

-> If so, then why do 'NSNotFound' and 'kCFNotFound' have different values on the Foundation and CoreFoundation respectively?

They don’t have the same value; we translate it manually in most places.

Also, I tested by modifying the 'kCFNotFound' value in CoreFoundation/Base.subproj/CFBase.h (See below code snippet):

if TARGET_RT_64_BIT
static const CFIndex kCFNotFound = LONG_MAX;
#else
static const CFIndex kCFNotFound = INT_MAX;
#endif

With this change the above mentioned test-case passes but a lot of other TestFoundation tests seem to fail.

Please provide your comments on the appropriate change that is needed here.

Thank you.

I think we probably just need to check the result from the CF function and translate it into an NSNotFound in this one particular case.

I think the problem in this case is that we fetch a buffer of CFRanges and pass that to the result creation.

ranges: UnsafeMutablePointer<CFRange>
...
let result = NSTextCheckingResult.regularExpressionCheckingResultWithRanges(NSRangePointer(ranges), count: count, regularExpression: matcher.regex)

Thanks,
- Tony

Regards
Mamatha
Mamatha Busi
<Mail Attachment.gif>
Java L3 Support
IBM Software Group
India Software Labs

e-mail: mamabusi@in.ibm.com <mailto:mamabusi@in.ibm.com>
Java Technology India Group <http://bit.ly/a7KpQj>

_______________________________________________
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org <mailto:swift-corelibs-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

_______________________________________________
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org <mailto:swift-corelibs-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

Mamatha_Busi · March 8, 2016, 12:15pm

@Philippe Hausler, @Tony Parker: Thank you for the response.

As per Tony's suggestions I have translated CF's kCFNotFound into an
NSNotFound of Foundation in 'NSTextCheckingResult' (the place where the
ranges are being constructed). Below is the code snippet with the changes:

_regularExpression = regularExpression
        super.init()
        let notFound = NSRange(location: NSNotFound,length: 0)
        for i in 0..<count {
            ranges[i].location == kCFNotFound ? _ranges.append(notFound) :
_ranges.append(ranges[i])
        }

@Philippe: Request your comments on the above code changes.

TestFoundation has been run and I see no failures with the above changes.

The tests 'test_complexRegularExpressions' in the TestNSRegularExpression
were excluded till date. I am now running the excluded tests as well and
there are a few test-cases here which deal with the ranges having
NSNotFound. Along with these should there be more tests written to test
the same?

Regards
Mamatha

Mamatha Busi

Java L3 Support

IBM Software Group

India Software Labs

e-mail:
mamabusi@in.ibm.com

Java Technology India Group

···

From: Philippe Hausler <phausler@apple.com>
To: Tony Parker <anthony.parker@apple.com>
Cc: Mamatha Busi/India/IBM@IBMIN, swift-corelibs-dev@swift.org
Date: 03/04/2016 12:02 AM
Subject: Re: [swift-corelibs-dev]
NSRegularExpression.firstMatchInString() fails on Linux
Sent by: phausler@apple.com

On Mar 3, 2016, at 10:22 AM, Tony Parker via swift-corelibs-dev < swift-corelibs-dev@swift.org> wrote:

Hi Mamatha,

On Mar 2, 2016, at 3:57 AM, Mamatha Busi via swift-corelibs-dev < swift-corelibs-dev@swift.org> wrote:

Hello

The following test case, that currently fails on Linux, is extracted from
some of the excluded tests inside TestFoundation/TestNSRegularExpression.

import Foundation

let searchStr = "123"
let testRegex = try NSRegularExpression.init(pattern:
"a(b|c|d)(x|y|z)*|123", options: )
let firstMatch = testRegex.firstMatchInString(searchStr, options: ,
range: NSMakeRange(0,3))

if NSEqualRanges((firstMatch!.rangeAtIndex(1), NSMakeRange(NSNotFound,0))
{
    print("Test passed")
} else {
    print("First match first capture range =
\(NSStringFromRange(firstMatch!.rangeAtIndex(1)))")
    print("Test failed")
}

In the above scenario, no capture group participates in the match. So,
firstMatch.rangeAtIndex(1) should ideally return range {NSNotFound, 0} but
the actual returned value is range {-1, 0}. Hence the failure.

Debugging on the CoreFoundation, it is observed that the value -1 is
coming from 'kCFNotFound' which is defined as -1. On the Foundation side,
'NSNotFound' is defined with the value of ‘Int.max’ (which must be
Int32.max and Int64.max on 32 and 64 bit platforms respectively).

There are a couple of queries that I have:

-> Is 'NSNotFound' in Foundation mapped to 'kCFNotFound' in
CoreFoundation?

-> If so, then why do 'NSNotFound' and 'kCFNotFound' have different values
on the Foundation and CoreFoundation respectively?

They don’t have the same value; we translate it manually in most places.

Also, I tested by modifying the 'kCFNotFound' value in
CoreFoundation/Base.subproj/CFBase.h (See below code snippet):

#if TARGET_RT_64_BIT
static const CFIndex kCFNotFound = LONG_MAX;
#else
static const CFIndex kCFNotFound = INT_MAX;
#endif

With this change the above mentioned test-case passes but a lot of other
TestFoundation tests seem to fail.

Please provide your comments on the appropriate change that is needed
here.

Thank you.

I think we probably just need to check the result from the CF function and
translate it into an NSNotFound in this one particular case.

I think the problem in this case is that we fetch a buffer of CFRanges and
pass that to the result creation.

ranges: UnsafeMutablePointer<CFRange>
...
let result =
NSTextCheckingResult.regularExpressionCheckingResultWithRanges(NSRangePointer(ranges),
count: count, regularExpression: matcher.regex)

Thanks,
- Tony

Regards
Mamatha

Mamatha Busi

<Mail Attachment.gif>
Java L3 Support

IBM Software Group

India Software Labs

e-mail:
mamabusi@in.ibm.com

Java Technology India Group

_______________________________________________
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

Mamatha_Busi · March 10, 2016, 7:39am

Hi Tony,

Thank you for the comments.

PR is created:
https://github.com/apple/swift-corelibs-foundation/pull/282

Regards,
Mamatha

···

From: Tony Parker <anthony.parker@apple.com>
To: Mamatha Busi/India/IBM@IBMIN
Cc: Philippe Hausler <phausler@apple.com>,
swift-corelibs-dev@swift.org
Date: 03/08/2016 11:11 PM
Subject: Re: [swift-corelibs-dev]
NSRegularExpression.firstMatchInString() fails on Linux
Sent by: anthony.parker@apple.com

Hi Mamatha,

On Mar 8, 2016, at 4:15 AM, Mamatha Busi <mamabusi@in.ibm.com> wrote:

@Philippe Hausler, @Tony Parker: Thank you for the response.

As per Tony's suggestions I have translated CF's kCFNotFound into an
NSNotFound of Foundation in 'NSTextCheckingResult' (the place where the
ranges are being constructed). Below is the code snippet with the changes:

_regularExpression = regularExpression
        super.init()
        let notFound = NSRange(location: NSNotFound,length: 0)
        for i in 0..<count {
            ranges[i].location == kCFNotFound ? _ranges.append(notFound) :
_ranges.append(ranges[i])
        }

@Philippe: Request your comments on the above code changes.

TestFoundation has been run and I see no failures with the above changes.

The tests 'test_complexRegularExpressions' in the TestNSRegularExpression
were excluded till date. I am now running the excluded tests as well and
there are a few test-cases here which deal with the ranges having
NSNotFound. Along with these should there be more tests written to test
the same?

Sure, more tests are always a bonus.

Please open a PR on swift-corelibs-foundatation when you’re ready to get a
review.

Thanks for tackling this!
- Tony

Regards
Mamatha

Mamatha Busi

<Mail Attachment.gif>
Java L3 Support

IBM Software Group

India Software Labs

e-mail:
mamabusi@in.ibm.com

Java Technology India Group

From: Philippe Hausler <phausler@apple.com>
To: Tony Parker <anthony.parker@apple.com>
Cc: Mamatha Busi/India/IBM@IBMIN, swift-corelibs-dev@swift.org
Date: 03/04/2016 12:02 AM
Subject: Re: [swift-corelibs-dev]
NSRegularExpression.firstMatchInString() fails on Linux
Sent by: phausler@apple.com

On Mar 3, 2016, at 10:22 AM, Tony Parker via swift-corelibs-dev < swift-corelibs-dev@swift.org> wrote:

Hi Mamatha,

On Mar 2, 2016, at 3:57 AM, Mamatha Busi via swift-corelibs-dev < swift-corelibs-dev@swift.org> wrote:

Hello

The following test case, that currently fails on Linux, is extracted from
some of the excluded tests inside TestFoundation/TestNSRegularExpression.

import Foundation

let searchStr = "123"
let testRegex = try NSRegularExpression.init(pattern:
"a(b|c|d)(x|y|z)*|123", options: )
let firstMatch = testRegex.firstMatchInString(searchStr, options: ,
range: NSMakeRange(0,3))

if NSEqualRanges((firstMatch!.rangeAtIndex(1), NSMakeRange(NSNotFound,0))
{
   print("Test passed")
} else {
   print("First match first capture range =
\(NSStringFromRange(firstMatch!.rangeAtIndex(1)))")
   print("Test failed")
}

In the above scenario, no capture group participates in the match. So,
firstMatch.rangeAtIndex(1) should ideally return range {NSNotFound, 0} but
the actual returned value is range {-1, 0}. Hence the failure.

Debugging on the CoreFoundation, it is observed that the value -1 is
coming from 'kCFNotFound' which is defined as -1. On the Foundation side,
'NSNotFound' is defined with the value of ‘Int.max’ (which must be
Int32.max and Int64.max on 32 and 64 bit platforms respectively).

There are a couple of queries that I have:

-> Is 'NSNotFound' in Foundation mapped to 'kCFNotFound' in
CoreFoundation?

-> If so, then why do 'NSNotFound' and 'kCFNotFound' have different values
on the Foundation and CoreFoundation respectively?

They don’t have the same value; we translate it manually in most places.

Also, I tested by modifying the 'kCFNotFound' value in
CoreFoundation/Base.subproj/CFBase.h (See below code snippet):

#if TARGET_RT_64_BIT
static const CFIndex kCFNotFound = LONG_MAX;
#else
static const CFIndex kCFNotFound = INT_MAX;
#endif

With this change the above mentioned test-case passes but a lot of other
TestFoundation tests seem to fail.

Please provide your comments on the appropriate change that is needed
here.

Thank you.

I think we probably just need to check the result from the CF function and
translate it into an NSNotFound in this one particular case.

I think the problem in this case is that we fetch a buffer of CFRanges and
pass that to the result creation.

ranges: UnsafeMutablePointer<CFRange>
...
let result =
NSTextCheckingResult.regularExpressionCheckingResultWithRanges(NSRangePointer(ranges),
count: count, regularExpression: matcher.regex)

Thanks,
- Tony

Regards
Mamatha

Mamatha Busi

<Mail Attachment.gif>
Java L3 Support

IBM Software Group

India Software Labs

e-mail:
mamabusi@in.ibm.com

Java Technology India Group

_______________________________________________
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev