Javascript: 클라이언트 단에서의 대용량 데이터 처리 성능 이슈 개선 # 비동기처리 # Chunk # requestAnimationFrame (rAF)

브라우저에서 request를 통해서 데이터를 받아와서 브라우저에 나타내는 과정에서

데이터가 굉장히 큰 경우 병목현상이 발생했다.

이 병목현상을 개선하는 예제를 정리

 

사용 예제

  인풋창에서 텍스트를 입력하면 해당 텍스트를 포함하는 유저를 실시간 검색한다.

 

병목 발생 지점

  1) request - SQL - response 과정에서 SQL문 자체가 오래걸림

  2) javascript data handling

     1)을 통해 받아온 데이터가 크다보니, javascript를 통해 UI에 데이터를 삽입하는 과정에서 병목 발생

 

해결방법

  1) request 비동기 처리

    - Ajax를 활용해서 request를 하고 있는데, request 및, response 처리 과정을 Promise를 사용하여 비동기처리로 변경

  2) UI 데이터 삽입시 전체가 아닌 분절하여 화면 삽입

    - 데이터 분절

    - 분절된 데이터를 삽입 (requestAnimantionFrame (rAF)사용)

       rAF 사용이유:

         - setTimeout, setInterval 등을 사용할 수 있으나, 클라이언트 단의 자원 사용량과 UX를 고려하여 rAF 사용

         - 자원 사용량: 브라우저를 벗어나면 rAF는 작업중단.

           이에 반해 setTimeout 등은 해당 브라우저가 비활성화 상태여도 프로세스를 진행함

         - 위 과정에서 쓸모없는 자원이 사용되기도 하며,

           브라우저가 다시 활성화 됐을 때 누적되어있던 프로세스가 한번에 반영되면서 UX를 훼손

 

예제 코드

const recipientInput = document.getElementById('recipient');

// 1) 비동기 처리 부분 - Promise 사용
function _searchMembers(data) {
    return new Promise((resolve, reject) => {
        $.ajax({
            url: '<?php echo G5_URL; ?>/jasamall/',
            type: 'POST',
            data: data,
            success: function(response) {
                const members = JSON.parse(response.data);
                resolve(members);
            },
            error: function (xhr, status, error) {
                reject(error);
            }
        });
    })
}

async function _fillSearchedMembers(data, tagId, ulId) {
    const members = await _searchMembers(data);
    const singleFieldTags = document.getElementById(tagId);
    const searchedMemberUl = document.getElementById(ulId);
    singleFieldTags.style.display = 'block';
    singleFieldTags.innerHTML = '';
    renderMembersInChunks(members, singleFieldTags, searchedMemberUl, 100); // chunk size can be adjusted
}

// requestAnimationFrame을 계속 실행 또는 중지할지 결정하는 flag
// true이면 중단, false이면 계속 실행
let abortChunkProcessing = false;

// requestAnimanaionFrame을 사용할 함수
function renderMembersInChunks(members, singleFieldTags, searchedMemberUl, chunkSize = 500) {
    let index = 0;

    // rAF에 사용될 callback 함수 (chunk process)
    function processChunk() {
        if (abortChunkProcessing) {
            return;
        }

        const end = Math.min(index + chunkSize, members.length);

        for (let i = index; i < end; i++) {
            const member = members[i];
            const li = document.createElement('li');
            li.innerHTML = `${getMemberIcon(member[0])} ${member[1]}(${member[0]})`;
            li.addEventListener('click', () => {
                searchedMemberUl.innerHTML += `
                    <li style="display: flex; align-items: center; background-color:#F3F5F7; border-radius: 5px; padding: 5px; margin-right: 5px; height: 25px;">
                        <span class="searchedMembers" style="white-space: nowrap" mb_id="${member[0]}">${member[1]}</span>
                        <button onclick="removeSearchedMember(event)" style="background-color: #F3F5F7; border: none; cursor: pointer;">×</button>
                    </li>
                `;
                recipientInput.value = '';
                singleFieldTags.style.display = 'none';
            });
            singleFieldTags.appendChild(li);
        }

        index = end;

        // 처리해야 할 members가 남아있으면 다음 chunk 프로세스를 실행
        if (index < members.length) {
            requestAnimationFrame(processChunk);
        }
    }

    // chunk 프로세스 시작
    requestAnimationFrame(processChunk);
}

function searchMembers() {
    const recipient = recipientInput.value;
    if (recipient === '') {
        return;
    }
    abortChunkProcessing = false;
    const alreadySearchedMembers = Array.from(document.querySelectorAll('.searchedMembers'))
                                    .map(member => member.getAttribute('mb_id'));
    const data = {
        recipient: recipient,
        alreadySearchedMembers: alreadySearchedMembers,
        action: 'searchMembers',
    };

    _fillSearchedMembers(data, 'singleFieldTags', 'searchedMemberUl');
}


recipientInput.addEventListener('input', () => {
    // debounce를 통해 2초 이내에 입력이 없으면 검색 요청
    clearTimeout(window.searchTimeout);
    abortChunkProcessing = true;
    window.searchTimeout = setTimeout(searchMembers, 500);
});

 

예제 커밋: https://github.com/Junanjunan/15sir/commit/95faa49ee6798a7f5f35255f4337677557253915

 

참조 블로그:

https://inpa.tistory.com/entry/%F0%9F%8C%90-requestAnimationFrame-%EA%B0%80%EC%9D%B4%EB%93%9C

 

ChatGPT 질문 내용

 

나:

It operates well. But I have question about rAF. Is there special reason that using rAF? can I substitute another way? And if it is, nevertheless there is another way, is rAF best way?

 

ChatGPT

Great to hear that your chunked processing is now operating well! Let's delve into your questions regarding requestAnimationFrame (rAF):

1. Is there a special reason for using requestAnimationFrame (rAF)?
2. Can you substitute it with another method?
3. If substitutions are possible, is requestAnimationFrame the best way?

1. Why Use requestAnimationFrame?

requestAnimationFrame (rAF) is a browser API designed to optimize animations and visual updates. When used in the context of chunked processing, it offers several advantages:

• Synchronization with Browser's Refresh Rate: rAF schedules your callback to run just before the browser repaints, ensuring that your updates are in sync with the display's refresh cycle. This minimizes visual glitches and jank.
• Efficient CPU Usage: Browsers can optimize rendering by batching multiple rAF callbacks, leading to better performance, especially for animations or frequent UI updates.
• Automatic Throttling: When the tab is not visible (e.g., minimized or in a background tab), rAF callbacks are throttled or paused, conserving CPU resources and improving battery life on mobile devices.
• Smooth UI Updates: By aligning your updates with the browser's rendering cycle, rAF ensures smoother transitions and interactions, which is beneficial when rendering large datasets incrementally.

2. Alternatives to requestAnimationFrame

Yes, there are alternatives to rAF for scheduling chunked processing:

a. setTimeout / setInterval

• Usage:

function processChunk() {
    // Process a chunk of data
    // ...

    if (moreData) {
        setTimeout(processChunk, 0); // Schedule next chunk
    }
}

setTimeout(processChunk, 0); // Start processing

 

• Pros:
  • Simple and widely supported.
  • Allows you to specify delays between chunks, which can help in managing processing speed.
• Cons:
  • Not synchronized with the browser's rendering cycle, potentially leading to less smooth UI updates.
  • May lead to less efficient CPU usage compared to rAF.
  • Potentially more latency in scheduling, especially with very small timeout values.

b. setImmediate (Non-Standard)

• Usage:

function processChunk() {
    // Process a chunk of data
    // ...

    if (moreData) {
        setImmediate(processChunk); // Schedule next chunk
    }
}

setImmediate(processChunk); // Start processing

 

• Pros:
  • Schedules callbacks to execute as soon as possible after the current event loop tick.
• Cons:
  • Non-Standard: Not supported in all browsers (primarily available in Internet Explorer and Node.js).
  • Lack of cross-browser compatibility makes it unreliable for web applications.

c. Promises with async/await and setTimeout

• Usage:

async function processChunks() {
    while (moreData) {
        // Process a chunk of data
        // ...

        await new Promise(resolve => setTimeout(resolve, 0)); // Yield control
    }
}

processChunks();

 

• Pros:
  • Cleaner syntax with async/await.
  • Allows yielding control back to the event loop, keeping the UI responsive.
• Cons:
  • Similar to setTimeout in terms of synchronization and efficiency.
  • Slightly more verbose compared to rAF for certain use cases.

d. Web Workers

• Usage:

// In main thread
const worker = new Worker('worker.js');
worker.postMessage(data);

worker.onmessage = function(e) {
    // Handle processed data
};

 

 

• Pros:
  • Offloads heavy processing to a separate thread, completely avoiding main thread blocking.
  • Ideal for CPU-intensive tasks.
• Cons:
  • More complex to set up and manage.
  • Communication overhead between main thread and worker.
  • Not a direct substitute for scheduling incremental UI updates.

3. Is requestAnimationFrame the Best Way?

requestAnimationFrame is often the most suitable choice for chunked processing related to UI updates because of its synchronization with the browser's rendering cycle. However, whether it's the "best" depends on your specific requirements:

When requestAnimationFrame is Ideal:

• Incremental UI Rendering: When you need to render large datasets incrementally to keep the UI responsive.
• Smooth Visual Updates: When you want to ensure that your updates are in harmony with the browser's repaint cycles, reducing visual stutter.
• Resource Efficiency: When you want the browser to optimize CPU usage by batching rAF callbacks.

When Alternatives Might Be Preferable:

• Non-UI Related Processing: If you're performing heavy computations that don't directly relate to visual updates, Web Workers might be a better choice.
• Controlled Timing: If you need precise control over the delay between chunks, setTimeout with specific intervals might be more appropriate.
• Browser Compatibility Constraints: If you need to support very old browsers that do not implement rAF, setTimeout can be a fallback.

Final Recommendation:

For your use case—rendering large arrays in the DOM incrementally while keeping the UI responsive—requestAnimationFrame is generally the best approach due to its synchronization with the browser's rendering process. It ensures that your chunked updates are efficiently handled without causing significant jank or lag.

However, it's essential to balance chunk size. Too large a chunk can still cause noticeable lag, while too small can lead to excessive scheduling overhead. Experiment with different chunk sizes to find the optimal balance for your specific scenario.

 

5. Conclusion

requestAnimationFrame is a powerful tool for managing chunked processing in a way that harmonizes with the browser's rendering cycle, ensuring smooth and responsive UI updates. While there are alternatives like setTimeout, they lack the synchronization and efficiency benefits that rAF provides, especially for tasks that involve frequent or large-scale DOM manipulations.

By implementing an abort mechanism alongside chunked processing, as shown in the enhanced example, you can ensure that your application remains responsive even when dealing with large datasets and frequent user interactions.

If you encounter further performance challenges, consider combining these techniques with other optimizations like DOM manipulation batching, leveraging document fragments, or even integrating virtualization libraries for managing large lists.

Feel free to ask if you have more questions or need further assistance!