The Gunners took an early lead thanks to an Alexandre Lacazette header but were pegged back by Michail Antonio before the break.
It took a winner from substitute Eddie Nketiah with just five minutes to play for Arsenal to grab the three points and keep up their 100% start to the season.
Arteta was thrilled at full time, but he is painfully aware that there is still an awful lot of work to do on his side if they are going to compete at the tip of the Premier League table.
‘I am really happy with the three points,’ the Spaniard told Sky Sports. ‘I need to analyse properly, because we have a lot of issues that we created ourselves in many moments.
‘It reminded me a little bit the game, and phases when we played them last year and let them run.
‘We created enough chances to win the game, but we conceded too may chances that were our fault.’
It was certainly a flatter performance from the Gunners than their opening day win over Fulham last weekend, with the Hammers having chances to produce the upset and taking twice as many shots as their hosts.
However, they got over the line and will go into their trip to Liverpool a week on Monday full of confidence.
‘What I like was the approach that the boys had in the last 25 minutes when I could see they were getting a little bit down, they brought it up and probably a few months ago we would probably have drawn or lost this game, but today we won it,’ said Arteta.