Abercrombie & Fitch chairman says some customers are too afraid to shop - and it's destroying sales

Abercrombie & Fitch just took a major hit.

The brand saw comparable sales decline 8% for this quarter, compared to a 9% decline this time last year.

The parent company, which includes a brand for children and Hollister, saw comparable sales fall 4%.

The main problem plaguing Abercrombie? Challenges abroad.

Chairman Arthur Martinez told Business Insider that the company was "disproportionately affected" by the lack of tourists in flagship stores and international stores. He also said people in European markets are worried about personal safety following recent terrorist attacks, and that that would deter them from going out and shopping.

He referenced that that terrorism fears, coupled with other factors, was "feeding into an air of caution" for consumers.

Wunderlich Securities analyst Eric Beder pointed out the company has been losing steam internationally earlier this week, too.

With foot traffic down, it's hard to get people into stores - which is where most of Abercrombie & Fitch's changes are happening.

"What we can control is what happens when someone walks across the threshold," he said.

But there are other current ugly trends in the apparel market that the brand will have to fight to win over consumers.

Right now, there's a trend towards tons of promotions, which has made it extremely hard for brands like Banana Republic and J. Crew to wean consumers off of purchasing apparel strictly at sale prices.

Abercrombie & Fitch is currently working to wean customers off of promotions, too; Martinez says the company needs to be"clear about what our long term objective is about reduced promotional activity and stay the course."

Additionally, Martinez highlighted how there aren't "any overwhelmingly powerful fashion trends in the market place" right now. That could make it harder for a brand to stand out if it doesn't have the right price point.

"That's a definitely a headwind facing the entire specialty retail space since 2012 when colored denim was a trend," Mizuho Securities Managing Director Betty Chen said to Business Insider. "Since then, there's been very little dominant trends and most retailers have struggled."

And, then, of course, there's Abercrombie & Fitch's reputation.

Chen pointed out that Abercrombie "is not fighting the brand heritage" and that "they're probably not fighting the perception of premium price point that A&F certainly had in the past," she said.

Though the company has the negative weight that its younger and more successful sibling Hollister does not, it has made some strides by toning down its sexual imagery and updating its style.

"This is a journey," Martinez said. "There won't be an "a ha!" moment where we pull the covers off something and say, 'here's the silver bullet answer.' Research tells us that there is a longing for the Abercrombie brand to become 'my' brand again. We hear quotes, like 'I've grown up but the brand didn't grow up with me.'"

A part of that is shifting its reputation as a teen retailer.

"I'd really like the world to stop referring to Abercrombie as a teen brand," Martinez said, highlighting how the reality is that most of Abercrombie's shoppers are actually in their twenties - not teens.

And ultimately, he thinks things are moving in the right direction, even if sales aren't saying that.

"The ... comp number [doesn't] look encouraging, but you have to look under the hood here [and see the] engine of this company, and there are more than a few things that are happening very positively," he said.

Still, he acknowledged that the next quarter would "remain challenging" but we should "expect to see improvement over the balance of the year." The company recently appointed Stacia Anderson, formerly of Victoria's Secret, to be the President of the brand.